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EDITOR* S NOTE 



While the publiishing of the proceedings of the Statewide 
Confer<ence on Institutional Research offers »il oppoirtiinity to 
demonstrate the vitality of the practitioi^er$ and programs with- 
in Qi^r institutions, the spirit of the conference itself is only 
minimally displayed on these printed pages. The enthusiasm with 
which the contributors presented their works to their colleagues, 
the follow-up of question and answers in formal and informal set- 
tings, and the formulation of new insights, notions, and ideas 
through relaxed professional interaction cannot be captured by 
this volume. 

Similarly, two innovative conference sessions, promulgated 
by the planning committee, proved to be quite successful to par- 
ticipants although inadequately represented by the published 
proceedings. 

In the first of these sessions. Dr. Charles Hoover of the 
National Institute of Education reviewed for conference partici- 
pants the workings of the ERIC (Educational Resources I^nformation 
Clearinghouse) publications system. The discussion reflected a 
duality of interest on the part of the institutional researchers: 
the comprehensive nature of the retrieval system which character- 
izes ERIC stimulated questioners exploring the potential for 
enhancements in daily institutional research activities, while 
at the same time participants were apprised of the relatively 
easy means which ERIC could provide for disseminating the signifi- 
cant information being produced by virtue of their professional 
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efforts. Although not designed as a primary thrust of the con- 
ference program, this "peripheral" session generated sufficient 
interest to warrant consideration for similar undertakings at 
future conferences . 

As an opening to the principal theme of the conference — 
external impacts upon institutional research — the planning com- 
mittee arranged a discussion session which included staff repre- 
sentatives of the education committee of both the Florida Senate 
and the Florida House of Representatives. Dr. Gus Tumbull, 
staff director of the Senate Education Committee, and Ms. Jane 
Hayman, legislative analyst of the House Education Coiranittee, 
outlined in some depth the results of the 1976 legislative session, 
drawing attention to the information needs requisite to passing 
sound legislation. For their part, conference participants ex- 
pressed their concern over the increasing demand for information 
of questionable importance which requires both expensive and time- 
consuming effort on the part of institutional researchers. 

By the way, a note of special appreciation is due to Archie 
Johnston of Tallahassee Community College for his assistance in 
providing a comprehensive written version of the discussion. 
(How could he have known that the novice editor would overlook 
that detail?) 

Despite the volatile nature of this never-to-be-resolved 
dichotomy, the candor and high level discussion which characterized 
the session allowed all participants to take satisfaction in know- 
ing that their needs, problems, and insights had been successfully 
conveyed to those who labor "on the other side of the fence." 
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With these considerations, and in view of the quality efforts 
contained in this volume, there should be little surprise that we 
look forward to the Tenth Annual Conference next June. 



Bill Law 
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XI 



Your I :r z slative Session — 1976 

Dr. Gus Tumbull, staff director of the Education Committee 
of the Florida ftjuse of Representatives, and Miss Jane Hayman, 
legislative analyst from the Florida Senate Education Committee, 
conducted a question and answer session on happenings of the 1976 
Florida Legislature. They were introduced by Dr. Craig Johnson 
of Florida State University and this report is prepared from a 
series of tapes and notes by Dr. Archie B. Johnston of Tallahassee 
Coiranunity College. 

Craig Johnson set the stage with a low-key presentation and 
a brief background of how the legislative session reflected the 
education community. Each presenter was asked to give about a 
15-minute talk of how he/she saw the session from the legislative 
floor and some of the back-room politics which led to various 
legislative decisions. Gus Turnbull spoke first. 

"There were a nxamber of interes'^iing bills which never passed, 
and some of rhem had dated back for a considerable period of time. 
They v'ill, more than likely, be back next year, as it nearly 
always happens that desired legislation may take several years 
to become law. The process of passing laws interests me greatly, 
and I find myself almost as much interested in the techniques 
through which some of these things happen as by the actual sub- 
stance of what eventually comes out^" said Turnbull. 

He pointed out that most of the audience are people who are 
used to the rational process of using data to provide information 



14 
1 



for specific purposcj, and he also expressed an interest in how 
to get that data to the appropriate agency. 

Turnbull offered to toss out several ideas very quickly and 
asked that if the audience were interested in specific legislation, 
he would follow through on that. 

He mused that perhaps our basic problem is that there very 
often isn't a legislative interchange and continued, "As I think 
on the title of our program this morning, it seems that too often 
we aren't bouncing our ideas off the other party, and as a result 
there is a real lack of understanding. Part of that is a basic 
difference between what you and the legislative personnel thought 
they were talking about during the legislative session. Many 
different interests come together from very different points of 
view about the nature of educational enterprise. There's also a 
very real difference of views, needs and pressures between you 
and the legislature. Those of you from institutions are naturally 
advocates of your particular programs, and you understand your 
program far better than the legislature. On the other hand, the 
legislature is exposed to many different interpretations, of 
which yours is only one. 

"You find it very hard to expand programs, or in these days 
just to stand still by running very fast, what with double-digit 
inflation, enrollment caps, the soc^ring cost of enrollment and 
the very little increase in dollars. On the other hand, legisla- 
tors are faced with similar demands from every element of state 
government. They are faced with very persuasive arguments as to 
why each program should be expanded even in this time of economic 



recession and inflation. They obviously have to listen to only 
one agency and forget all the others or treat them all equally 
by cutting them back, or make the pie bigger by raising taxes. 

"This session we found out about the pressure against rais- 
ing taxes from the people back home. The legislators listened 
to them, not to the agency advocates. The message was coming 
through fairly logically and clearly that people did not want to 
be taxed. 

"Looking specifically at education, there are some veiry 
real differences in i:he concept of how education should be 
governed. On the proper role of the legislatvire and the proper 
role of educators, parents, students, and the public at large, 
there is not a consensus. Perhaps at one time there was. In 
Florida more than in other states we have had, I think, a much 
greater tradition of (I'll have to use the word) political 
interference.' No, let me change that to political 'involvement* 
in education. In Georgia, for example, there is a different atti- 
tude, as illustrated in the political caunpaign now going on in 
Athens, where a candidate for the state legislature involved him- 
self in a university issue. He was promptly jumped upon by the 
other candidates who said that it was totally out-of-bounds for 
a legislative candidate to involve himself in the internal affairs 
of the university system. We have seen in Florida, over the years, 
a large variety of Florida legislators and other government offi- 
cials who have involved themselves very deeply in the internal 
affairs of the universities and the junior colleges. There is 
not only a difference between the states but a difference of 
viewpoints from within the state as to how and in what way govern- 
ment should become involved in educational decisions. 
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"Be that as it may, every active group is concerned with 
winning support, but the process is not always clear. A little 
example that you may have noticed is the problem of increasing 
tuition this year. The university presidents were unanimously 
against fee raises and almost every legislator spoke out against 
raising tuition, but somehow tuition got raised. 

"There is also a concern with conflicting values which we've 
seen highlighted in some special legislation. The conflict this 
year which became probably the most heated topic in educational 
legislation wa? the idea of using a lottery in a graduate admis- 
sion process — absolute merit versus relative merit. Then there 
are some continuing and very real administrative policy problems: 
how do you govern a nine-campus university system? How do you 
govern 28 community colleges? And what happens when the legis- 
lators are faced with these two very different managerial struc- 
tures? In trying to react to them, they face many of the same 
problems in two different systems with very different structures. 

"The legislature is, of course, very concerned with the cost 
of academic programs. How do you balance the part of the income, 
which comes from everybody, with the benefit that the individual 
students are getting? How much should the students themselves 
pay? There is a very real lack of solid information; there i" a 
very sparse amount of evidence as to what is actually happening 
out there in the institutions. The problems resulting can be 
resolved partly by legislation and partly by efforts of groups 
like this. A visible sign of progress is the whole managerial 
approach to government of education and information systems. 
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"Now, given all these differences and problems and concerns, 
I think that in order for either party to communicate with the 
other, we need to become much more familiar with each other's 
processes. Both of us tend to think that the other is in posses- 
sion of a "white box" where we put the problems in one end and the 
answer comes out the other. Yet within that "box" is a whole host 
of strange and wonderful inter-relationships. 

"In this last session the Education Committee had over 200 
bills to worry about. The impact of particular personalities 
and the power of individuals and structures within the legisla- 
ture are vital in determining which bills finally get onto the 
floor. The values held by the individuals that serve these com- 
mittees are important. It was no accident, for example, that a 
large number of education bills were never actually heard. There 
was a definite feeling against passing a lot of education bills. 
At least we on the House side {and I don't want to put Jane on 
the spot by asking her to comment on it) heard this time last year 
that in 1976 there wouldn't be much educational legislation because 
the Senate's tired of it. The rumors were true. Some 35 bills 
which had passed the House, after going through our committee, 
died in the Senate, not because they were voted down, but because 
they were never heard. 

"One has to mderstand such details to understand the pro- 
cess; likewise those of us on the legislative side who look to 
the educational community and expect to get simple answers are 
going to be surprised. We discover fairly quickly that there may 
be a university system, but there are still several different 
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points of view within that system. And then there are 28 com- 
munity colleges which don't pretend to be a system, although 
the legislature sometimes tries to treat them that way, U'lder- 
Jtanding the educational process is as much or more difficult 
than understanding the legislative process- 

"I think another thing we've got to do if we're going to 
coiranunicate effectively is to gain some confidence in each other 
as individuals and as institutions. Too often there is a tendency 
to draw a mental image of the other and react to that image in- 
stead of to reality. 

"To shift back to the educators' side, on a long range basis 
one must do a good job of impressing the students. A lot of the 
negative reaction which education has experienced with the legis- 
lature comers d.lrectly from students. Students who have graduated 
and gone on to become legislators themselves, or aides or committee 
staff members, go back to specific things which happened to them 
as they went through that educational process. Often these were 
things that they didn't like, and now they are suddenly in a . 
positioi^. to express themselves and have an impact. In addition, 
the student, lobby as such is becoming active. In some respects 
they've had a better success ratio than the professional educa- 
tional lobby. 

"Another important part of coimnunication is to learn how to 
state our differences objectively, without getting into a person- 
ality conflict. We have to understand the reasons why viewpoints^^^ 
and particularly those coming from different perspectives, are 
going to differ. It doesn't vrtaan that either side is banal or 
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corrupt or immoral or even stupid. There is a tendency, perhaps 
more on the academic side than on the legislative side, to assume 
that because the individual has taken a negative side that he must 
be a bad person. Consequently, we can't trust him or work with 
him. Yet good /legislators and good lawyers disagree very intensely 
and profoundly on an issue and then they turn around the next day 
and work together on a different issue. These differences on the 
issues do not detract from their personal relationships. In order 
to work with the legislators you need to develop this approach. 

"Perhaps the last thing I should mention in dealing with im- 
proved communications is one that that is particularly appropriate 
for this group, and that's providing accurate information in a 
timely fashion. That's something that is very important. Really 
the easiest way of all to earn the confidence of the other party 
is to provide an accurate answer to a question very quickly. 
Having done this three or four times you can get away with almost 
anything. But quite frankly, the record of the educational com- 
munity to being responsive to this kind of data request is not 
very good." 

-k a 
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EA/EO: INSTITUTIONS, STUDENTS, POLITICS, AND FAIRNESS* 



Scarvla &. Anderson 
Educational Testing Service 



4/12/76 - An end to open admissions. ..was approved last week 
by the governing board of the City University of 
New York (Magarrell, 1976b/ p, 5). 

3/29/76 - In Florida's 28 community colleges, the traditional 
"open door" has closed. College officials estimate 
that 5,000 qualified students have been turned away 
since last fall (Magarrell, 1976a; ^p. 9). 

12/22/75 - For the poor, access to California's four-year 

public colleges "remains very limited," according 
to a new legislative study (Slevert, 1975,. p. 13)* 

11/3/75 - At the Champalgn-Urbana campus of the University of 
Illinois, enrollment for the spring term has been 
closed in an effort to reduce total enrollment by 
1,000 students (Magarrell, 1975a,. p*l). 



11/3/75 - Campuses using oil and natural gas are warned they 
can expect unit price to have quadrupled since 
1969-70 (Magarrell, 1975b, p.S). 



Thus does The Chronicle of Higher Education chronicle some of the 
conditions that are causing educators and legislators. to take a new look 
at the concept of "equal access/equal opportunity." Social scientists 
(e.g., Dressel, 1976; Willingham, 1974; Messick, Note 1), for their part, 
are examining the fundamental relationships among equity to students, 
adverse impact on specific population groups. Institutional and program 
efficiency > and benefits and risks to society. And discovering some basic 
incompatibilities, as we shall see. 



*Keynote address. Ninth Annual Florida Statewide Conference on Institutional 
Research, Orlando, Florida, June 17-18, 1976. I am grateful to William I. Sauser 
Jr. , for help with the documentation for this paper* 
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The reminders are dramatic that this Is a multivariate world — Including 
the segment called "higher education" — and univariate models > principles » or 
slogans lead to disappointment and dismay. 



Institutional Purposes 

Let us begin by looking at student access and opportunity In relationship 

to the nature of the college or university. The major Issues here revolve 

around the question of "how closely the goals of [higher] education should be 

shaped by the requirements of the job market" or, as Manning (Note 2) has put 

it, "to what extent is education little more than job training puffed up with 

a great deal of windy, vague and high sounding rhetoric." (p,16). On one hand, 

we have what might be characterized as a classical view pf higher education — 

a knowledge-oriented view. According to Ebel (Note 3): 

Excellence in education. . .is primarily excellence in 
learning. What is learned is mainly verbal knowledge. 
Knowledge is assimilated information. . .',.The power of 
knowledge in human affairs is so plain, and attempts 
to cultivate it are so pervasive in the enterprise that 
the failure of some educators to acknowledge its primacy 
is hard to understand. 

Then Ebel goes on to ask why such acknowledgment from educators is lacking: 

Is it because they see that the possession of knowledge 
does not guarantee wisdom or virtue (or success in the 
job) in all who seem to possess it? 

Is it because the pursuit of knowledge has been the 
traditional focus of educational efforts and they 
want to break new ground? 

Is it because knowledge is often hard to gain, and 
they are looking for something that is easier and 
more fun to do? 

Is it because these educatv:>rs have had somethiug 
less than remarkable success in getting possession 
of much of it? 
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Even among those who accept the primacy of knowledge In the higher 
education enterprise, there Is some disagreement about direction. For 
example, Kreps (Note 4) has said that "small liberal arts colleges are 
characterized by disseminating knowledge and universities by generating 
knowledge." 

To knowledge, Bok (1974) and others (e.g. ^ Dressell and Mayhew, 1954; 
Mattfield, 1974; Muller, 1974) have added other important objectives of 
higher education including "acquisition of skills and habits of thc^ght — 
such things as the ability to write effectively, to read with comprehension, 
to analyze problems, .to understand quantitative methods. • .and, indeed, even 
to understand an academic discipline sufficiently well as to gain a sense of 
what it means to master at least one subject in depth" (Manning, Note 2, p.21). 

As we mention skills areas we are getting closer to the focus of some 
of the postsecondary programs that some think are better characterized as 
training than as education. At one extreme, as Manning (Note 2) has pointed 
out, administrators may turn "to the Dictionary of Occupational Titles for 
the significant principles that should guide higher education prograiud" and 
describe the products of their institutions as " trained manpower , rather than 
educated people" (p. 22) . 

If we look at an April 26 article In Newsweek (Shells, McGee, Boyd, & 
Monroe, 1976), we find Boston University president John Silber saying, "It^s 
about time we got off the backs of youngsters who don't have the aptitude for 
college or the interest in it" (p. 64). This statement is offered in support of 
the general proposition that "the growing respectability of alternatives to col- 
lege is viewed by most educators as a healthy development" (p. 64). "On the other 
hand," the Newsweek authors note, "many are deeply disturbed by the capitulation 
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to careerlsm on college and university campuses" (p. 64). "The Inordinate 
pursuit of professionalism leads to much narrow-minded bigotry," says Father 
Daniel O'Connell, president of St, Louis University, "The word in German is 
accurate and applicable," he continues • "it is Fachidioten — specialty 
idiots" (p. 64). Dean Rosovsky of the Harvard Faculty of Arts and Sciences 
sums up some of the confusion this way: "Every year at graduation, we say 'We 
welcome you to the community of educated men and women, ' This should mean 
something, but at the moment it doesn't mean anything. It may mean that you've 
designed your own curriculum, it may mean that you know all about urban this or 
rural that. But there is no longer a common denominator" (p. 64)* 

William Turnbull (Note 5), president of ETS, in a discussion of admis- 
sion standards in higher education, introduced what he calls "the fallacy of 
higher education as monolith~the fallacy that there is a common set of stand- 
ards accepted in American higher education" (p. 14). Not only must we agree 
with Turnbull that American higher education lacks a common set of standards, 
but it is clear that it lacks a common set of educational goals and objectives 
as well. I, personally, think that this is sensible — and sensitive to the 
needs of a changing society. However, we are obviously not going to resolve 
the debate about the purposes of higher education — multiple or otherwise — here. 
What we must recognize is that existing variations in goals and standards for 
higher education have their counterparts in variations in the meaning of equal 
access/equal opportunity. At the most pragmatic level, we cannot determine, 
for example, whether an open admissions program has achieved success unless, in 
Trivett's (1976) words, "prior agreement is reached on the purposes served by 
that program" (p. 3). 25 
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But there Is a more fundamental problem: How can Institutions fairly 
offer access and opportunity if they cannot provide prospective students 
with a clear definition of what the access is to and the opportunity is for ? 
In this age of "consumer protection" when we insist on publishing the ingre- 
dients of pet foods and warning of the hazards of cigarettes, our catalogs 
provide a very incomplete, and frequently misleading, picture of what going 
to college will really be like. And this is especially true for students 
whose family dinner conversations have not included discussions of what 
college was like for mama, papa, or older sisters and brothers. In fact, the 
percentage of "middle-class males who are opting for traditional higher edu- 
cation has dropped from 44 percent in 1969 to 33 percent this year — and most 
of the dropouts are white" (Sheils et al,, 1976, p.61). But this new realism 
does not seem to be equally matched in other population groups which still see 
college primarily as the great door opener. Again according to Newsweek, 
"working-class whites and blacks continue to see college as the only open window 
to the better life [we're mixing our metaphors on modes of egress here!], and 
women see degrees as essential if they are to have careers" (p«61). 
Student Characteristics 

So equal access and equal opportunity are student-confounded as well as 
programr-confounded concepts. Let's look specifically at some of the charac- 
teristics of students that may — or should — influence access/opportunity policies, 
as well as at the way those policies may have a differential impact on different 
students. I have already mentioned one important student characteristic that 
suggests different approaches to recruitment and retention — that is, student 
expectations both about what higher education is like and what purpose it will 
serve in their lives. 
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I have enjoyed talking vith Frederick W. (Bi!.!) Schutz (Note 6) who, among 
other things, spearheads the affirmative action program for engineering students 
at Georgia Institute of Technology. By any objective standards, Schutz' s pro- 
gram can be called a success. For example, the increase in both numbers and 
retention rate for Tech engineering students who are female, from minority 
groups, and even from states other than Georgia must greatly exceed anything 
that an "old grad" could envision.. A great deal of Schutz' s strategy is based 
on either changing the expectations of potential students or helping to bring 
them into line with reality. First, he says, he has to find students who have 
taken courses that provide relevant background for the engineering courses 
Tech offers. That part is relatively easy, and he starts with such rosters as 
those provided by the National Society of Professional Engineers' Minority 
Engineering Education Effort and the College Board Student Search Service, as 
well as specific referrals. But then, having identified likely candidates, he 
has to convince some of them that it's xrorth che extra effort engineering school 
requires and others that they're more frightened of engineering school than they 
need to be. A challenging problem of diagnosis and treatment'. Furthermore, 
when the recruitment process begins depends on the sex of the candidate. Girls 
and — I hate to repeat what Schutz says here — especially Southern girls frequently 
need to be convinced that being good in math is OK and a career in science or 
technology is a possibility for them^ and they need to receive these messages 
no later than the 8th or 9th grade. [Julian Stanley (Note 7) makes this same 
point with respect to girls in his program for mathematically precocious youth.] 
On the other hand, recruitment of boys, including strong involvement of personnel 
from hometown engineering firms, can wait a couple of grades. With black students 
— and their parents — sometimes the need is to stimulate consideration of 
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engineering as an alternative to the ministry, education, law, or medicine, 
which have traditionally been regarded as the high-status professions by the 
black community. 

I have not done justice to the Georgia Tech program herr,. It is more 
compreli <2nsive and less coercive than my remarks indicate. In addition, it 
is a very special case, involving relatively able students and a career field 
that is not badl> plagued by unemployment. However, the program does illustrate 
some of the difficulties and subtleties in making equal access/equal onportunity 
a reality. 

In many cases, students are so conditioned by their backgrounds that they 
do not even seek opportunities in higher education. Although the overall sta- 
tistics with respect to college enrollment of black students have improved 
remarkably since the Coleman report (Coleman et al., 1966) — from less than five 
percent in 1966 to nine percent in 1974 (U.S. Bureau of the Census, 1975) — we 
know that the range of choices for black students is still more restricted than 
the range of choices for vliite studv<%nts. For example, black students are still 
more likely to attend institutions in the "state college" rather than the "state 
university" category (Conrad & Cosa^.d, 1976, p. 27; Institute for the Study of 
Educational Policy, 1975). The situation seems to be even more serious in the 
case of students from Spanish-American backgrounds (e*g., see Ferrin> Jonsen, & 
Trimble, 1972). Furthermore, proportionately larger nimibers of minority than 
majority students drop out of high school before graduation. And even the — to 
us — meaningless attendance diploma provides more likelihood of access to 
postsecondary education than no high school diploma at all. 

By the way, it is time that we quit apologizing for blaming schools for the 
poor quality of the students who enter many institutions of higher education. 
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Schools ought to be blamed for sending forth students with poor verbal and 
quantitative abilities — deficiencies that, when we get down to fundamentals, 
are the greatest deniers of all of equal access and equal opportunity. But 
the circle is vicious, for the colleges were the ones who trained the school 
teachers and administrators in the first place. And colleges continue to 
give As aad Bs in English to a majority of the freshmen students who 
professors complain "can't write" (Maeroff^ 1976, p. 42). 

Before we leave consideration of the students who may or may not seek 
equal access/opportunity, we should note a few other student characteristics 
that are relevant to admissions, recruiting, and retention decisions: 

Age and marital statu3 . On the average, college students are 
getting older. In two-year institutions, for example, the average 
age has shifted from 22 only three or four years ago to 28 now (Cohen, 
Note 8, p. 5). Some are delaying entrance; others are coming back. 
More are married. Different life styles from those of traditional 
undergraduates are probably reflected too in increasing ratios of 
part-time to full-time students and increasing numbers of s;: ocial or 
"unclassified" students. These phenomena introduce both new dimensions 
to equal opportunity and a new criticalness about the opportunity 
afforded. For example, consider this "Ms." question: Are we allowing 
equal opportunity to a married woman student if we cannot offer 
admission or employment for her husband? 

Ability . Julian Stanley used to say that he had never seen 
a child — rich or poor; black, white, or brown — who read "two grades 
over level" who was described as "disadvantaged" (Anderson, 1973, 
p. 204). Certainly ea/eo policies have very different impacts on 
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minority or women students who score In the 700s on the 
SAT and those whose scores are closer to 250. Ability 
factors make a profound difference even in who elects to 
submit to the SAT, ACT, and other admissions procedures 
in the first place* 

Economic condition and geographical locat^ .-.i. Some • ' 
potential students are too poor even to take advantage 
of full scholarships (Conrad & Cosand, 1976, p. 29). And 
prospects for ever increasing tuition costs are staggering 
(Fleming, 1975). Furthermore, the availability of reasonably 
convenient public postsecondary institutions is highly 
variable across the states. Florida students are luckier 
than many in this respect. 

Such differences in the targets of ea/eo policy render any unilateral 
policy ipso facto unfair to some candidates. As Dressel (1976) has pointed 
out, what we have here is a problem of ••fit." Basically, colleges and 
universities have three models to choose from: 

1. They can define programs, establish prerequisites for success 
in them, and select students who are likely to be successful — in other 
words attempt to match the programs and the students . 

2. They can define programs, establish prerequisites for success 
in them, admit a wide range of applicants including some who do not have 
the defined prerequisites, and Include as part of their endeavor efforts 

to provide the "deficient" students with the necessary competencies through 
tutoring, remedial programs, counseling, etc. — in other words, try to fit 
the students to the programs . 
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3. They can leave their program definitions relatively loose, admit 
a wide range of applicants, and develop and adapt programs to the abilities 
and interests of the students — in other words, try to fit the program to 
the students ♦ 

Political Pressures 

I have listed these options in order of cost, which leads us directly 
into some of the political problems and pressures associated with atteii?>ts 

to provide more equal access, and opportunity* The open admissions program 

at CUNY was not scotched because of overwhelming research evidence about its 
merit or lack of merit. It was scotched, as Harrington (1975) predicted it 
would be some months before in the New York Times Magazine, for politico- 
economic reasons. In Harrington's opinion, the demand to charge tuition, 
which accompanied arguments for closing admissions, was "a symbolic gesture 
designed to convince the American money market that Hew York City has given 
up its sinful, innovative ways" (p.l6)* William F. Buckley (1976), hardly an 
advocate like Harrington of the CUNY program, noted:* "It is instructive that 
the mess is being accosted not by the application of reason.. ♦But by that face- 
less, leaden, brute force: money" (p.23-A). 

New York City is a special case, to be sure, but few individual colleges 
or university systems are playing the game of "unlimited resources" these days. 
And few campuses are without their professional or community rivalries which 
might end up being fought out in the ea/eo arena. Having seen standardized 
tests become the whipping boy for all sorts of unrelated educational frustra- 
tions, I am well aware that the object of attack may have little to do with 
the causes. This is not to say that some ea/eo programs and the battalions 
surrounding them may not deserve annihilation. If they are conceived with- 
out conviction Hnd implemented without humanity, they may do juo;re harm than 
good. And in some cases, money may better be spent on a new cafeteria than on 
remedial classes. All I am urging here is that admissions officers, recruiters, 

18 31 



and directors of special ea/eo programs be alert to the competing pressures 
on their institutions, and try to manage their ea/eo programs — and assemble 
evidence of their validity—in light of the competition. 

Perhaps the assembly of such validity evidence could become a focus of 
efforts by institutional research centers. The director of the ERIC 
Clearinghouse for Junior Colleges (Cohen, Note 8, p. 2) noted recently that they 
had "found institutional research to be concerned excessively with student 
flow— where they came from, where they went" and to be using "incredibly 
simplistic" designs. Maybe evaluation of alternative ea/eo-related programs 
and policies would offer the IR community more scope. This brings us back 
around the full circle, of course: You cannot evaluate ea/eo-related programs 
and policies without attempting to delineate the purposes of the college or 
university programs, appreciate the characteristics of the likely student 
population to be served, and analyze the competing politico-economic pressures 
on the institution. 

Fairness 

Equal access/equal opportunity is not isomorphic with open admissions, 
of course (Conrad & Cosand, 1976, pp. 21-24). In fact— and in spite of the 
messages I read at the beginning of this paper — many colleges place few 
delibera*-e restrictions on admissions. For example, an unpublished survey 
of members of the National Association of State Universities and Land Grant 
Colleges showed that 33 — or about a third — of the 96 members reporting "had a 
policy of admitting any graduate of an accredited high school within the state 
in their 1971-72 admissions" (Trivett, 1976, p,3). But what has happened is that 
these same institutions also enjoy a great deal of student "drop oiit" and 
"ship out" especially in the first year. Freshman math was the chief villain 
at the land grant college I attended. 

32 




19 



In this discussion, I have also suggested that because of such factors 
as lack of encouragement, poverty, or earlier "missteps," some students never 
reach the point where they can take advantage of open admissions. And we can 
think of situations where easy admission of students to incompatible or 
inappropriate programs may actually denigrate or delay opportunity. However, 
admissions policies are certainly associated with access and opportunity in 
the public's mind. And there are still a great many institutions which because 
of their objectives, status, or facilities can— or must — be selective. Most of 
these institutions are probably doing a good job of recruitment and admissions 
with most of their students. Equally probably, however, they are not doing 
quite as good a job as they would like to do. The gap may relate to the state- 
ment in the introduction of this paper about the relationships among equity, 
adverse impact, efficiency, and societal risks. So it may be well to examine 
that statement further — in the context of a higher education program that has 
more applicants than places. 

Let us think first about equity. And let us assume that we have an admis- 
sions system (including perhaps academic ability measures, previous grades, 
ratings of motivation and the like) which truly predicts success in a program- 
to the extent that success can be defined and predicted. (It is unfortunate 
that in too many real situations we have had to make such an assumption without 
very good evidence to back it up.) Now we have a group of candidates ranked 
from high to low in terms of the admissions criteria. Then doesn't the most 
equitable procedure seem to be to select the students in order of rank — regard- 
less of race, sex, national origin, relationship to a wealthy alumnus, or profi- 
ciency in tennis? Unfortunately, because of historical conditions — ^which many 
of us feel guilty about but few can be held diri>ctly responsible for — such a 
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procedure will almost certainly result in selecting disproportionately fewer 
students from certain population groups—for example, women if math scores 
are an important criterion, Cuban Americans if English language scores figure 
substantially in the ranks. In other words we cannot maximize equity for all 
students and nonadverse impact on certain population groups at the same time* 

Now suppose that we adopt some compromise between equity for all students 
and admission of a larger proportion of students from some population groups, 
including some fairly "high risks." In good conscience, weM like to help the 
low members of this group as much as possible to avoid failure and frustration. 
This probably means remedial instruction, personal and financial counseling, 
special programs geared to their interests, etc. And we are now into insti- 
tutional/program efficiency. It does not seem possible to maximize both lack 
of adverse impact and management efficiency. 

Finally, consider the nature of the programs being offered and their 
relationship to society. Can we afford to admit less qualified students, 
provide them with less-than-optimal education and training, and then foist 
on the public less- than-compe tent primary teachers, nurses, bridge builders, 
or electricians? Perhaps, if we continue to turn out relatively poor college 
instructors in certain fields it will go unnoticed. But certain professional 
and service careers are crucial, and we cannot afford for the products of our 
effort to put society at risk — no matter how internally well-^meaning and 
efficient those efforts are. 
Summary • 

I'm afraid that this speech has rather ranbled over many aspects of the 
equal access/equal opportunity problem and provided few answers. Thus, it is 
not easy to try to summarize my remarks. However, my favorite Yazoo City 
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English teacher. Miss Omle Parker, always said that speeches must have 

a summary — especially after lunch. So I believe that these are the four 

main points I want to make: 

First, I am reminded that the tragedy of T. H. White's King Arthur, 

as some have analyzed it, was that goodness, virtue, and even mystical inter- 
vention were not enough. In the area of ea/eo, goodness is certainly not 
enough . 

. So, second, we have to employ all of our ratloclnative processes 
in disentangling the complicated interactions among Institutional/program 
purposes, student expectations and other characteristics, and politico-economic 
pressures on institutions. Such analysis is essential to developing sound 
ea/eo policies, implementing them, and evaluating their effects. 

. Third, in the admissions process, it is probably impossible to maximize 
fairness to individuals in terms of equity, to population groups in terms of 
lack of adverse Impact, to ins tl tut ions / pro grams in terms of efficiency, and 
to society in terms of benefits. Designers of -ea/eo programs must be ever 
conscious of the tradeoffs involved, arid must be prepared to compromise — and, 
in some cases, sacrifice one good for another. 

. Finally, in spite of the emphasis here on the complexity of the 
ea/eo area — of the difficulties involved in interrelating institutional, 
student, political, and ethical considerations — we have made remarkable progress 
in providing access and opportunity in higher education. And this is true 
whether the standard is our own past or the efforts of other nations. As we 
strive — as we must — to improve our programs, we are entitled at least every 
once in a while to pause and contemplate our successes. 
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THE WACT OF THE EQUAL EDUCATION 
AND EQUAL OPPORTUNITY MOVEMENT ON 
UNIVERSITIES AND THEIR MANAGEMENT 
INFORMATION SYSTEMS 

Abstract 

This paper discusses the impact of the equal opportunity movement ^ 
on universities and the management information systems they use to produce 
and analyze required data. The variety of federal rules, regulations and 
legislation that have been promulgated to attack the problem of discrimi- 
nation are outlined. The mixed emotions of the university community 
toward the numerous regulatory guidelines are examined in regard to the 
general impact on universities. The expensive and complicated process 
of having to adapt the management information system of universities to 
meet the d eman ds of federal requirements is featured. The paucity of 
information, specific data requirements, the results of system develop- 
ment, and the challenge for the future are each included in the discussion. 
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THE IMPACT OF THF EQUAL EDUCATION 
AND EQUAL OPPORTUNITY MOVEMENT ON 
UNIVERSITIES AND THEIR MANAGEMENT 
INFORMATION SYSTEMS 

According to a recent article in the Atlanta Constitution , it would 
take 50 professional baseball stadiums to hold all the crumpled-up 
paperwork generated in one year. If you do not bother with the crumpling 
and just stack all the forms one on top of another, the resultant stack 
would be three times as high as the Washington Monument (Martin and 
Townsend, 1976). There can be little doubt but that a large percentage 
of all this paperwork is directly related to federal policies designed 
to end racial and sexual discrimination. There can be no doubt but 
that all the paperwork required for monitoring and reporting on policies 
dealing with discrimination has had a fundamental impact on universities 
and their management information systems. 

Federal Regulations and Legislation 

A variety of federal rul;iS, regulations and legislation have been 
promulgated, each of which attacks the problem of discrimination in a 
different manner. Before attempting to addredis the question of the 
impact of this effort to end discrimination, it is important to have a 
clear understanding of the scope of the effort. 

There are four sets of federal regulations and laws that now cover 
discrimination in institutions of higher education. 
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1- Executive Order 11246 as Amended by Executive Order 11375 
Executive Order 11246 is not a law, but a series of rules and 
regulations that must be followed by institutions that have accepted 
federal contracts. Its purpose is to prevent discrimination in 
employment, institutions holding federal contracts are required to 
file with the Department of Health, Education, and Welfare a written 
plan of affirmative action to "remedy the effects of past discrimi- 
nation" and to prevent the continuation of current discrimination 
(Sandler, 1974, p. 2). 

2. The Equal Pay Act of 1963 as Amended by Title IX of the Education 
Amendments of 1972 — The Equal Pay Act was the first sex discrimi- 
nation legislation. It requires that employers raise the salaries 
and pay back-pay to nonprofessional women who have been found to be 
underpaid as compared to their male counterparts. Title IX of the 
Education Amendments extends the coverage of the Equal Pay Act to 
executive, administrative, and professional employees, including 
all faculty (Sandler, 1974, p. 2). 

3. Title IX of the Education Amendments of 1972 (Higher Education Act) — 
Title VI of the Civil Rights Act of 1964 originally spoke to the 
issue of discrimination on the basis of race, color, and national 
origin in all federally assisted programs. Title IX extends the 
provisions of Title VI to include sanctions against discrimination 

on the basis of sex. All educational institutions, whether public 
or private, that receive federal money from grants, loans, or 
contracts are forbidden to discriminate on the basis of sex, as 
regards both students and employees (Sandler, 1974, p. 3). 
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4, Title Vn of the Civil Rights Act of 1964 as Amended by the Equal 
Employment Opportunity Act of 1972 Title VII of the Civil Rights 
Act forbi s discrimination on the basis of race, color, national 
origin, religion, and sex in employment. The Equal Employment 
Opportunity Act amends Title VII to cover all educational insti- 
tutions, public cr private, whether or not they receive any federal 
funds (Sandler, 1974, p. 3). 

The four sees of federal regulations and laws present institutions 
of higher education with a wide range of complex and often tjmes confusing 
requirements. All aspects of employment are covered, including hiring 
practices, working conditions, salaries, promotions, and terminations. 
Most practices regarding students and student programs are now also 
covered. 

General Impact on the University 

There is no question but that most universities support the concepts 

of equal education and equal opportunity for all. However, the impact 

of the numerous regulatory guidelines that have been imposed is viewed 

by the university cornriunity with mixed emotions. 

In the spirit of the Bicentennial, the presidents of the four major 

private universities in the nation's capital have published "A 1976 

Declaration of Independence," in which they have expressed the concerns 

of many a university both public and private: 

"During the first two centuries of this nation's life, govern- 
ment leaders were dedicated to the proposition that institutions 
of higher education were independent, voluntary associations 
serving public and private purposes. By being so dedicated, 
governments at all levels encouraged colleges to create a system 
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of quality and diversity that was to become the standard of 
academic excel lance throughout the world. 

"Recent government policies and behavior toward education, 
however, have threatened this valued independence and have 
shaken the foundations of our system of higher education in this 
country. 

". . . With fifty administrative agencies and two dozen 
committees of the U.S. Congress . . . having direct responsi- 
bilities that impinge on our universities, governmental regulation 
and oversight can become a disguised form of governmental control. 
Institutions are driven to defensive strategies. 

"The innovative c. d searching analysis expected of colleges 
of society suffers in the face of the mounting necessity for 
dealing with the myriad, pedantic, and sometimes contradictory 
requirements imposed by government regulation. Diversion of 
faculty and staff attention to questions of compliance is a 
damaging intellectual cost which universities and society at 
large can ill afford to pay." (Cleary, Walton, Elliott, and 
Henle. 1976) 

The slow and subtle erosion of its philosophical foundation as 
suggested by Cleary, et. al. , may in the long run prove for higher 
education to be the most fundamental impact of the equal education and 
equal opportunity movement. A more immediate, more obvious and certainly 
more desirable effect relates to the changes universities fiave made in 
recognizing many of the basic human rights. 

Fundamental changes have been made in recruitment practices. 
Universities now employ elaborate procedures for advertising faculty and 
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staff vacancies in journals and periodicals that are li'.sly to be circu- 
lated among women and minorities. Student recruitment has also been 
changed to entice more women and minorities to enroll. Special financial 
aide, counseling, and academic tutorial programs are now integral parts 
of the student affairs program at most universities. 

Employment practices have been changed to reflect the ne'^ds of women. 
Maternity leave benefits have been implemented or made more realistic. 
Nepotism rules have, for the most part, been abolished. Retirement and 
insurance policies have been revised. 

These changes and the myriad of others that have occured-have resulted 
in a rocond major effect that relates to the cost of such programs to 
universities. In this time of financial exigency, cost, above all else, 
has captured the immediate and critical attention of the university 
community. 

Johnson (1974, p. 61) writes that '*The U.S. Department of Health, 
Education, and Welfare has made it clear that the affirmative action plan 
negotiated between Berkeley and HEW will serve as a model for all other 
affirmative acition plans negotiated between institutions of higher 
education and HEW." Universities of similar size and complexity as 
Berkeley, using the Berkeley model, might then expect to incur costs 
similar to those of Berkeley. For 1974-75, the State of California 
appropriated $250,000; the Regents appropriated $500,000; over $2 million 
was spent by absorbing costs within regular institutional resources; 
and another $500,000 was spent from internal funds specifically ear- 
marked for affirmative action ~ for a total of approximately $3 million 
{Johnson, 1974, p. 61). 

In the State of Florida, $1.57 million was appropriated in 1975-76 
for the State University System "Plan for Equalizing Educational Opportunity." 
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This amount is exclusive of any of the funds spejil, by_any„of_jthe._„ _ 
universities in negotiating and implementing their own affirmative action 
plans or programs i 

Impact on Management Information Systems 

The cost of data production and analysis represents a substantial 
percentage of the high cost of the equal education and equal opportunity 
programs. The process of adapting the management information systems 
of universities to meet the demands of federal requirements has proved 
to be expensive and complicated. 

Originally the major problem confronting universities related to 
the lack of basic data. The paucity of information deprived federal 
officials and campus administrators of the facts needed for policy 
formulation and for day-to-day decision-making. 

In search of meaningful data, under the auspices of Executive 
Order 11246, federal officials issued a comprehensive set of speclflcar 
tlons for employee data. Universities were required to establish a basic 
employee data file. The file was required to contain, at a minimum, 
fourteen basic data elements': 

1. name and/or identification number 

2. sex 

3. ethnic identification 

4. year or date of birth, or age 

5. current salary (full-time annual equivalent) 

6. current job family or generic job family 

7. current job title 

8. personnel action resulting in current job title (new hire, 
promotion, transfer, demotion) 
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9. date of personnel action resulting in current title (years 
in current job) 

10. previous job title 

11. employment status (full-time, part-time, tenured, non-tenured) 

12. educational level 

13. organizational unit where employed 

14. date of hire (U.S. Department of Health, Education, and 
Welfare, 1972, p. J-2). 

Universities were required to develop the capability to produce 
analysis on the basic data in the following manner: 

1. By department, a list of each job classii'ication -in descending 
order showing the numbers by sex for each racial and ethnic 
group, as well as cumulative figures for minorities and for 
females generally. 

2. icb classification, within the entire institution, showing 
the numbers by sex for each ethnic group, as well as cumulative 
figures for minorities and for females generally. 

3. By department, the mean salary^ in each job classification, by 
sex for each racial and ethnic group. 

4. By job classification, across department lines, the mean salary 
in each classification, by sex for each racial and ethnic group 
(U.S. Department of Health, Education, and Welfare, 1972, 

p. J-2). 

Specifications for student data, that are as detailed as those for 
employee data, have not yet been issued. Universities are finding, however, 
that to meet reporting requirements they are having to develop compre- 
hensive data bases of student data. Critical data elements include: 
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1. name and/or identification number 

2. sex 

3. ethnic identification 

4. year or date of birth, or age 

5. nation of citizenship 

6. level of highest degree 

7. student's classification level 

8. student program/program category 

9. current term course load 

10. level of degree granted 

11. grade point average 

Development of the systems required to generate and maintain the 
data that is being required has involved the expense of computer hard- 
ware and software, keypunch operators, programmers, machine time, and 
report production. The cost of the university personnel who develop 
and maintain the manual records that serve as the original source of 
data and of those who use the data to produce the variety of statistical 
reports required has also proved to be a considerable expense (Johnson, 
1974, p. 62). 

In 1974, after four years, Mary Lepper, of the Office of Civil 
Rights, reported that there had been some improvement in the amount and 
quality of data available, but that there was need for further improvement. 
The data on women was better but those on minority groups remained 
inadequate (Fields, 1974). Now, two years and millions of dollars later, 
further improvements have been- made, but the challenge remains. 

Now that universities have developed relatively reliable systems 
for generating and analyzing the data that is required, the future holds 
the prospect of dealing with the costly effects of continually changing 
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federal definitions and reporting requirements. The challenge to the 
university community will be to develop new methods and new systems which 
will substantially reduce the costs of monitoring and reporting on 
policies and procedures dealing with the equal education and equal 
opportunity movement. 
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ABSTRACT 

Recent reports of state and federal government agencies make it 
clear that students and their families are now generally accepted as 
the primary consumers of educational services and are entitled to the 
consumer protection concepts and mechanisms guaranteed by law. 
Implicit and explicit, in the mechanisms of consumer protection, is 
the concept of full disclosure; the idea that consumers are better 
protected when they have full access to meaningful product information 
and have full knowledge of their "consumer rights" and responsibilities. 
As the lack of full disclosure has prompted regulatory agencies into 
seeking protection for the student/family consumer in higher education, 
attention must now be given to the determination of relevant 
institutional data necessary to . 5 ^i;?d:^nts and families in raking 
informed and intelligent decisions concerning educational opportunities. 

The purpose of this paper is to identify information to assist the 
prospective student in evaluating programs and institutions of higher 
education through the development of a reporting format, a "consumer's 
guide," that lends itself to easy analysis and interpretation. In 
addition to cost, retention, and placement success information usually 
suggested by consumer protection groups, the consumer's guide will in- 
clude data for institutional and program evaluation, focusing attention 
on the data necessary for student consumer admissions decisions. 
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A NEW CAVEAT: LET THE BUYER BE AWARE* 

Larry G. Jones, The University of Georgia 
In spite of the genera! reluctance on the part of educators to 
recognize or accept the academic and marketplace parallels, economic 
theories and concepts are finding their way into education with in- 
creasing regularity and intensity. Consumerism, the latest of the 
concepts from the marketplace to be applied to education, gives the 
promise of a new reckoning for educators with the theoretical and 
practical intrusions from the world of commerce. 
Consumerism and Education 

There are several reasons to believe that the age of the consumer 
has come to education whether or not educators are ready and willing to 
accept Consumerism as an operating principle. First, Consumerism, the 
protection of the consumer from inequitable treatment by the vendor, is 
a cause or right many will champion and few will argue against 
(Arnstein, 1974). Second, the student, because of his time, energy and 
financial investment, has won the uncontested role of postsecondary 
educator's primary consumer (Shulman, 1976; Willett, 1975; El-Khawas, 
1975). Perhaps the most obvious reason that consumerism will become an 
educational as well as an economic concept, however, is the fact that 
consumer advocate groups and state and federal governmental agencies 
have already developed rules, regulations, and legislation to protect 
the student as a consumer (Department of Health, Education, and Welfare, 
1975; Educaiton Commission of the States, 1973; Shulman, 1976; Willett, 
1975; Peterson, 1970). 

* This paper was also presented at the Annual Forum of the Association 
for Institutional Research, May 1976 

52 

39 



While some academicians, true to their calling, will debate the 
definitions and others will fret over the appropriateness of a 12-month 
or 12,000 mile guarantee on college degrees, or recalling graduates 
with "defective parts" or other cynical analogies to "consumerism," the 
real concern of educators should be in correcting the abuses which make 
the student a consumer in need of protection (Dykstra, 1966). The fact 
is that nowhere has the old caveat, "Let the Buyer Beware," been more 
assumed than in education. At least in business there was the 
countering, if not often prevailing, attitude of "The Consumer is 
Always Right." Unfortunately, there are now generations of educational 
consumers who have come to believe, if not know, that the "student is 
never right." The question is no longer whether the concepts of con- 
sumerism will apply to the relationship between the student and the 
postsecondary institutions, but rather how the student will be assured 
of his consumer rights. 
The Basics of Consumerism 

Consumerism is based on the philosophical conviction that the 
consumer and the vendor meet in the marketplace as equals (Willett, 
1975). 

Equitable treatment for the consumer is embodied in four basic 
areas of consumer rights, and it is the protection of these consumer 
rights that the consumer movement is all about (Education Commission 
of the States, 1974): 

1. The right to safety or protection in the purchase of a good 
or service; including the right to hold accountable the 
person or organization to whom the consumer pays his money. 
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2. The right to choose; including the right to participate In the 
decision making process which establishes the relationship 
with the vendor; 

3. The right to be heard; which implies a system for negotiation 
and arbitration of disputes over the consumer vendor relation- 
ship; .. 

4. The right to be informed; to have access to all pertinent 
information which might affect the consumer's decision to buy; 

While each of these rights is important and has significance in 
the consumer movement it is the right to be informed - full disclosure 
as it is also known - that has come to be regarded as the focal point 
of consumer protection, more likely than not because it has been lack 
of information which has caused problems in consumer safety, decision 
making and protection (Department of Health, Education, and Welfare, 
1975). It is on the premise, generally regarded as true, that con- 
sumers are best protected when they have full knowledge of their 
"consumer rights" and responsibilities, that the new caveat "Let the 
Buyer Be Aware" is proposed for the postsecondary education market- 
place. 

Full -Disclosure and Postsecondarv Education ' 

Without apparent exception, all of the reports, conferences, and 
regulations to implement consumer protection for students include 
recommendations for the release of complete and accurate information 
about the institution to the student or prospective student which 
might influence the student's decision to attend or accept offers from 
a college. The implication of the various recommendations, and the 
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. general conclusion, of severai.inYestigations, has -been^^..t^ 1 ittle 

attention has been given to the information needs of the student^ 
(Education Commission of the States, 1975; Fund for the Improvement of 
Postsecondary Education, 1974, Jung, 1975, Department of Health, Edu- 
cation, and Welfare, 1975; Willett, 1975; Education Comnission of the 
States, 1974). 

Unfortunately, most of the information now available to students 
is lost in the public relations rhetoric of catalogs and recruiting 
brochures. Other pertinent information never finds the printed page 
and is released reluctantly, if at all, only upon insistent direct 
questioning. In fact, one gets the impression from reading most 
college publications that "hard data" is systematically avoided 
(Dykstra, 1966). 

Everyone knows, of course, that decisions about where to go to 
college and what to study are probably more emotional than rationajl. 
But then, given more obj: ..tive information on which to base the 
decision the student may be less prone to rely on subjective factors 
(Dykstra, 1966). In any event, the student's decision must not be 
based on insufficient institutional data. 
Consumer and Information 

The reverse side of the assumption that consumers are wise enough 
to make good decisions if they are provided with accurate information 
is the awareness that the complexities of today's products - and 
product decisions - make it virtually impossible for the average con- 
sumer to become expert enough to fully utilize the information, 
accurate though it may be, to protect himself in the marketplace 
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- (Education Corimlssion of the states, 19^^^^ The inability of the 
consumer to interpret complex product information or to make meaningful 
comparisons or decisions on the basis of the information has prompted 
some to suggest that the government will have to assume the function of 
"interpretor" for the consumer (Education Commission of the States, 
1974). 

The frightening prospect of further government involvement in the 
reporting and interpreting of educational data coupled with the 
obvious need for more meaningful educational information for the 
student consumer, prompts this proposal for A Consumer's Guide to 
Education . 

A Consumer's Guide to Education 

The proposed Consumer's Guide to Education deals only with the 
identification of information for ti^e prospective student to use in 
evaluating programs and institutions, and the development of a 
reporting format that lends itself to easy analysis and interpretation 
by the student. The purpose of the Guide is not to rate institutions, 
but it should facilitate institutional and program comparisons. Much 
of the information the Guide will present is probably already 
collected and reported in one form or another - although probably not 
in one place - while other information may present some collection 
prolplems for many institutions. The illustrative material which 
follows, while not exhaustive, is intended to be an example of the data 
that could and should be reported by the institution in a straight- 
forward manner in A Consumer's Guide to Education . Particular interest 
is paid here to the data relating to full disclosure of academic 
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Pi'ogranis and to related student, faculty cost and resource information: 
areas which are most subject to insufficient data and misinterpretation. 
Institutio nal Identification and Directory Data . In addition to the 
current official corporate name of the institution, and the location 
of the main campus and the name of the parent institution if a branch 
campus, a complete chronology of critical institutional dates 
(founding date, first coursework, first degrees, name changes, etc.) 
should be provided to the student consumer. 

Institutio nal Control. Affiliation and Corporate Status . Although the 
eight basic types of institutional control used by the U. S. Office of 
Education are generally recognized as being descriptive for reporting 
institutional control, an explanation of what "control" means is needed 
as not all states, municipalities, religious groups or corporate en- 
tities "control" in the same fashion. This is particularly true for 
private institutions where terms such as "affiliated" or "associated 
- with" or "sponsored by" which have no clear or commonly understood 
meaning, are used to describe a college-church relationship. Pro- 
prietary schools should state all corporate relationships, parent 
corporations, holding companies, etc. 

Composition of Governing Boards . While it may be informative to have 
an actual listing of names of people on the Governing Board of the 
institution, it would be more helpful to have a statement of the 
special qualifications required for board membership, how the members 
of the board are selected, and what constituencies they represent. It 
is often possible to get a better reading on the "control" of an 
institution by knowing how its Governing Board is selected, than by 
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knowing its public, private, or corporate status. Proprietary schools 
should list the owners and/or principal stockholders. 
Institutional Accredit ation. Licer.aes. on Approval . The statement of 
institutional accreditation must include which of the six regional 
accrediting associations has recognized the institution, and the type 
of accrediting (full, candidate, 4 year college accredited" as junior 
college, provisional, covered under the regional accreditation of 
parent institution) awarded to the institution by the association. 
Note should also be made of the highest program levels offered, and 
specifically those which have and have not been accredited or approved. 
Schools not covered by the accrediting associations must state the 
authority, licensure, or approval under which they offer educational 
programs. 

Institutional Purpose, Objectives, Goals . The statement of 
institutional purpose, objectives, or goals, should be clearly stated 
and easily understood. It is recommended that the purpose, objectives 
or goals be stated in a style which is easily interpreted into 
specific outcomes. The student should be fairly certain after reading 
the statement of goals what he can be expected to do as a result of the 
educational experience. 

Institutional Evaluation. Institutions should state the methods and 
procedures it will use to determine whether or not the stated 
institutional purpose, objectives or goals are being met. Obviously 
the methods and procedures of evaluation will have to be tied to the 
stated purpose, objectives and goals, hence the previous suggestion of 
behavioral ly stated outcomes. 
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In addition to the reporting of institutional self evaluations; — 
educational audits or evaluations by outside groups (including 
accrediting reports) should be available to the student. 

In one form or another, the institution should state how well it 
is performing its mission, and how that performance has been deter- 
mined. 

Requirements/Qualifications of Faculty and Administration . An in- 
dication of the ability of the faculty and administration to accomplish 
the objectives of the institution is an important aspect of 
institutional evaluation. If specific requirements and qualifications 
other than those usually expected for an academic appointment (such as 
church affiliation) are required of faculty or administrators, they 
should be stated. Statistical data summarizing the qualifications 
needed for faculty and administrators to accomplish the stated 
institutional goals should be reported. Inmost cases the summary 
should include the highest earned degrees of the faculty. 
Other Faculty Data . Statistical tabulations of faculty and 
administration data should include the number of faculty by rank and 
function, length of service at the institution, age distributions, 
tenured faculty, and tabulations by sex and race. It is assumed that 
a complete listing of faculty, including their qualifications and 
their responsibil itie?,, will be available to the student, although 
probably in another document. 

Curriculum and Program Data . The Consumer's Guide to Education is not 
intended to replace the college catalog as a source listing of 
courses and course descriptions, or the detail of major and 
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institutional requirements for degrees V There" i s an obvibuOeea Th^^ 
ever, to make the information in the catalog closer to reality. The 
resemblance between course description and course content is too 
often only coincidental, and the courses listed in the catalog should 
only include those that have been taught In the last severaf years, or 
have an honest chance of being taught in the near future. ' 

The Guide would include a statement of the calendar type (quarter 
semester, 4-1-4, trimester, etc.) and the approximate dates of the 
beginning and end of each term. Semesters, for instance, can be quite 
different in concept depending upon when they start and end. Infor- 
mation on majors and academic programs should include: the type of 
professional accreditation held, if any (in addition to regional 
accreditation); the number of courses offered in the major area and 
the number of those courses taught during the last year; the total 
number of faculty qualified in the area and the full-time equivalent 
faculty teaching in the major during the past year; the degrees 
awarded or certification offered for completion of the major or 

■ 

program; the number of hours or courses required in the major for 
completion; and, the average number of years (or terms) needed to 
completa the program or a degree in the major. 

Information on majors and academic programs should also include 
the number of declared majors; the, number. of recent graduates or 
program completers; the basic ability levels; and the grade point 
averages of students in each major or program; and averages on "exit" 
tests. 
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Student-Data .- Some important student -data-hasal ready been-descri bed 
with the curric'iium and major program data, but clearly more 
information on student characteristics needs to be provided. Strange 
as it may seem, information other than raw totals or averages on 
students is generally not available. The following data should be 
provided. 

Enrollment s Enrollment by sex, race, age,, full-time, part-time, 
matriculation status, residence, and class should be provided for the 
most recent fall enrollment period. 

Socio-Economic Characteristics . Descriptive data in the socio- 
economic background of the student body, generally available from ACE, 
ACT or SAT summaries on entering students, should be reported for 
prospective students. 

Admission Requirements . All of the requirements and consider- 
ations for admission should be clearly stated, including the dates and 
deadlines for application and acceptance. The amount of any fees, 
nature of any fees or deposits and fee refund or waiver policies should 
be clearly stated. 

Ability Levels of Entering Freshmen . Frequency d7Stribi;tions of 
the admission test scores, high school grade point averages, or Qth^r 
ability measures or indices, should be presented along with the mean 
and median scores for entering freshmen. If regional, national, or 
similar institution averages are available they s/^ uld be reported as 
well . 
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Predicted Student Success . First quarter predicted grade point 
averages are frequently used in the admissions process, and when they 
are the prediction equation should be made available to the student. 
In addition, it would be possible to prepare a chart which would allow 
an easy approximation of the predicted grade point average for the 
prospective student. 

Student Achievement . Grade distributions by class, major, or 
department and student class rank by grade point average is information 
that could give the prospective student a notion of student attainment 
and institutional expectation. Simple distributions of the data should 
be provided. 

Student Retention/Persistence . Data on the number of students who 
start and complete programs at the institution, along with figures on 
the number of students who enroll from previous quarters should be 
available in the Guide . In addition, reported student reasons for 
leaving the college should be provided. 

Ability Levels of Program Completers . Frequency distributions of 
the same ability criteria used in admissions and reported for entering 
freshmen should be made for the students who have successfully completed 
their programs or degrees. Obviously the student's scores won't change, 
but the difference in the distribution scores for entering and exiting 
students can be revealing. 

Measures of Program Completer's Achievement . Senior comprehensives 
and "exit" exams of various types are coming back into vogue as the 
issues of program and institutional evaluation and accountability 
become more important. The institution needs to state the methods and 
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procedures it uses in evaluting the programs if offers, and report the 
results of the measures it uses to determine the success of the pro- 
grams, including the scores of its graduates on various nationally 
normed tests by program, major, or degree. 

Where appropriate, success rates on bar exams, medical boards, 
licensure tests, admissions to graduate and professional schools, 
etc., should be reported. 

Placement a nd Alumni Data . Closely related to the data on the 
achievement of program completors is the information about what 
graduates do after college. Proprietary schools face more demands 
for such information than do colleges and universitites, but the de- 
mand for more job related information for graduates is increasing for 
all postsecondary institutions. The information on placement of 
graduates should reflect those who have positions, those who have 
positions in their field of study, those who don't have positions, 
those who don't have positions but are looking for positions, those 
who aren't looking for positions, and those who intend to make a 
career of their present work, as well as information on how their 
college work relates to their employment. 

Tuition, Fees and Costs . In view of the fact that most institutions 
are seeking more than a one-year commitment from the student, it would 
seem only reasonable for the institution to project the anticipated 
cost to the student over the entire length of the program. Obviously 
tuition and fees will be subject to change, but the changes in fees and 
tuition aren't as unexpected or unplanned as most institutions would 
have students believe. (If increases are unplanned or unexpected that 
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says something about the quality of financial planning at the 
institution). Guaranteed tuition plans are all but gone now, but a 
projection of anticipated costs over a two or four year period need 
not be a cost guarantee, nor a commitment to single levels of cost 
over the period projected. What is needed is an indication of what 
the cost increases are likely to be, and those cost projections are 
or should be a part of the planning of all institutions. 

In addition to the tuition, activity fees, deposits, etc., that 
are required of all students, any special fees associated with a given 
program, e.g. music fees, uniforms, laboratory fees, must all be 
listed. As the special fees are often buried in the catalog, all cost 
data in the Guide would be reported together. 

In those cases where the student is required to room and/or board 
in institutional housing, special note should be made so the student 
will include that cost in his planning. 

Estimates of costs of books and supplies and any personal living 
expenses should be provided based on the averages for currently en- 
rolled students. Such data is used in determining financial aid at 
most institutions and should be reported. 

Institutional Finance . For the curious or cautious student, a complete 
current audit report of the institution should be available for 
examination: most students don't need that much information to get a 
feel for the solvency of the institution. However, basic financial 
information reported to all students should include a statement of 
revenue and expenses showing dollar amounts and percentages for each 
of the typical revenue and expense classifications and a balance sheet, 
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Particular emphasis needs, to be placed on the percentage of revenue 

generated by tuition, and the percentages of expenses going toward 
instruction. Someday, perhaps, auditor's statements (and accreditation 
reports) will tell the public more than that the books and records are 
kept according to accepted standards, and report specifically on the 
financial position of the institution. 
Student Rules and Regulations , ^ArrthiraiglT^lrinay^ 

give details of dorm house rules, etc., specific rules and regulations 
affecting the student's rights to continue his enrollment should be 
clearly stated. It is also important that the methods and procedures 
cf setting student rules and regulations be spelled out in detail and 
that the student knew the procedures of being found guilty of breaking 
the rules, the consequences, and the appeal procedures. 
Conclusion 

More likely than not, this first attempt at A Consumer's Guide to 
Education is a bit like a twelve year old's first reading of Eyerythinc i 
You Wanted to Know About Sex but Were Afraid to Ask : It's more than he 
wanted to know even if he had known what to ask in the first place. 
The point is, that as educators and researchers, we have known more 
about education, institutions, and how students mix with both than we 
have told or than has been asked. We need to find a way to get that 
knowledge to the student without turning them to total abstinence or 
complete promiscuity- 
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Footnote ' 

notable exception is the work being done by the Fund for the 
Improvement of Postsecondary Education through their project on better 
information for student choice of college. 
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Abstract 

The Response System with Variable prescriptions (RSVP), a 
Computer-Based Instructional Management System is described and 
presented as a means for individualizing instruction for small as 
well as large classes. The paper discusses the necessity for the 
front-end work to be undertaken by the faculty in order to utilize 
the RSVP System creatively and meaningfully. This instructional 
management and prescriptive system has great promise for enhancing 
the educational process and is limited only in relationship to the 
energy and creativity levels of the faculty who wish to use the 
system. Research related to the use of RSVP in "open learning" in 
Miami -Dade Coimnunity College is presented. 
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Introduction 

For the past quarter of a century, America has introduced 
each decade with a new theme for higher education. In the 
1950' s, the theme could have been "accent on selection." 
In the 1960's, it was "accent on access." For the 1970 's 
It is Accent on Learning , (cross, 1976; p, 1x) 
Prefacing thus, her recent book entitled Accent on Uarting . 
Dr. Patricia Cross presents a vivid description of the "silent" rev- 
olution that is taking place in institutlrTis of h learning. The 
"New" students in these institutions have been the ^..^etus for the "silent" 
revolution that demands changes in Instruct f'ona" .aiyns and teaching 
strategies. Accordingly, the 1970's have witnessed a variety of alter- 
nate education programs. 

Baskin (1S75) claims that the alternate educition programs "are 
designed t.;. provide new options and new ways to learn to persons whose 
learning needs were not being met by the nations campus-based system of 
higher education." Baskin classifies these programs as: 1. Certification 
of acl ievement (credit/exam); 2. Media and/or technology c?!:.isred pro- 
grams which focus on alternate modes of delivering information to students 
outside the classroom; and 3. Learner participation and experiential ly 
oriented programs. 

The second category of alternate programs presented by Baskin is 
more popularly known as "open learning." The authors of Diversity by 
Design (1973) aptly expressed the spirit of the concept of "open 
learning" when they Jtscribed it as "a philosophy of learning, a craft 
of teaching, a vision of life." 
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At Miami-Dade Community College, the Division of Open College has 
assumed the arduous task of implementing open learning rc'jrses. To 
date, eleven different college credit courses rme. bG.^n offered through 
television, radio and print materials. Intert^/ining these various 
components of instruction into an integrated learning program is RSVP 
(Response System with Variable Prescriptions), a Ccmputer-Bas^d Instruc- 
tional Management (CBIM) system.! Additionally, and more importantly, 
RSVP brings alivs instructor-student dialogue:^ Vn open learning, which 
is generally perceived as devoid of such interaction. No douot, the 
dialogues find expression in the written, rather than in the verbal form. 

It is the writers' contention that "teaching v;ith coinputers" Is a 
vay of thinking and a way of behaving. IS c:es call for teaching skills 
that differ in kind and amount frcim those needed in classrooms. Moving 
from on-tri :-spot, spontaneous, vocal dialogues to pre-planned, systematic, 
written di^:ogues is, doubtless, an alteration of teaching behavior that 
is not instantaneously achievable. Hopefully, what we have to say in the 
following pagers vrill lend credence to our contention. 
D escription of RSVP 

RSVP is a system of nine computer progr? ns which assists an in- 
structor to individualize instruction while managing up to 5000 students 
in any single course whether the course is taught on campus or 1n a 
remote *:.etting. RSVP operates by maintaining three fi ies for each 
course — course file, student file and student index file. Tne student 
file contains student records which include information on student 
characteristics and performance. 
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RSVP scores student responses (Response System) and prints student 
reports (Veiriable prescriptions) . The variability in feedback to 
students is achieved through the use of control mechanisms called 
decision rules which can utilize attribute and performance information 
to differentiate the students for variable prescriptions. The instruc- 
tor selects the criteria for differentiating students and composes the 
corresponding messages and prescriptions. In executing the instructor's 
commands, RSVP captures invcl liable data for research related to student 
learning and prescriptive teaching. 
Instruction via RSVP 

The instructional functions of RSVP are best utilized when the 
teaching-learning process is conceptualized as taking the form of 
Instructor-Student dialogues. The y ^mework of these dialogues con- 
sists of three stages: 

1. Instructor presentation of instructional stimuli of various 
types to elicit student response in objective response format; 

2. Student response to the instructor's stimulus presentation by 
choice of response options; and 

3. Instructor responses to the choice made by the student. 
Over the term, teaching a course does require each of the stages 

of the dialogue outlined above. In using RSVP it is highly desirable 
that the stimulus situations provided for student response be planned 
in advance together with possible options for student response and 
appropriate instructor prescriptions for student choices. Although it 
is possible to devise stimulus situations, options for student response, 
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and prescriptions as the term progresses, it can become hectic and 
finally a race to determine the strength and stamina of the instructor. 
It can be done! It is, however, profitable and des;irable to concep- 
tualize stimuli with prescriptive responses as the course is developed 
so that the objective response format items may be given to students 
when the course is started. 

Presentation of stimuli (for use w.th RSVP) may be by nearly any 
means: lecture, discussion, reading assignment, film, video-tape, 
audio--tape, field visit, laboratory work, project assignment, etc. — 
and the purpose may be either instructional or evaluative. The focus 
of this discussion is on the instructional aspects of the objective 
response format used with RSVP. Student responses to the stimuli pro- 
vided by the instructor are of importance, not only in the case of the 
correct answer, but also in the case of the choice of an incorrect 
answer or in some instances, where there is no correct answer for 
student choice. 

In construr.ting the stimulus situations for eliciting student re- 
sponses, there need to be clearly identified purposes for presenting 
the stimulus situation. Some of the purposes that lend themselves for 
meaningful instructional dialogues through RSVP can be: 

1. To identify (-he gaps in acquired information. . .gaps may be in 
the form of "uninformed" and/or "misinformed" condition. 

2. To check if students comprehend what they see, hear and read. 

3. To identify the difficulties a student may be experiencing 
while participating in the learning activities. . .difficulties 
may be inherent to learning and/or external to learning. 
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4. To eiicit the kinds of questions and/or doubts that a student 
may raise while participating in the learning activities. 

5. To elicit students* opinions, interests, reactions, endorse- 
ments, objections, etc., to what is being presented to them 
through learning activities. 

6. To challenge students' thinking with problem solving situations 
which are an extension of what has been presented. 

7. To encourage students to see several pieces of information in 
an inter-related context. 

Once the purpose is clear, stimulus situations can be presented 
in different ways, to follow or precede or occur during the learning 
activities such as reading, film viewing, class discussion, etc. As 
the options for student response are selected for a stimulus situation, 
prescriptions for that situation should be thought out. Interestingly 
enough, when an instructor attempts to prescribe, the flaws in the 
stimulus situation and options for student responses become apparent. 
When nothing more than "your answer is wrong, look up page n_, in your 
text" can be said by way of prescribing, one cannot but re-examine the 
question written in the first place. This awareness has led several 
of the instructors, who hrU»e \\S',id RSVP for instruction, to report that 
RSVP has helped them to fat-eorRo better teachers. 

Methods of prescribing can take many forms: prescription for each 
option, prescription for each item, prescription for a group of items, 
prescription in relation to attainment of course objectives, prescrip- 
tion by present and past performance, prescription according to student 
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attributes and interests, prescription in terms of student time schedule, 
location and on and on. Enumerating can become endless. It is done 
only to point out the possibilities. These forms are not mutually ex- 
clusive. Several forms can be used in conjunction. No two instructors 
need to use the same methods of prescribing. In other words, each 
instructor programs RSVP to suit his/her objectives for individualizing 
instruction. The crucial question is: What criteria should be selected 
to provide variable prescriptions to meet the individual needs of 
students? What better opportunity can an instructor ask for to express 
one's creativity and ingenuity to make teaching unique and meaningful? 
Research Related to RSVP 

Like all educational innovations, RSVP needs to face up to the 
rigors of evaluation. Beginning modestly with one course, but a large 
enrollment of 1163 in Winter of 1972, Open College now offers 11 differ- 
ent courses, attracting approximately 2,000 enrollments per semester. 
Since its inception and through the Fall semester, 1975-76, these courses 
had been offered to approximately 13,000 enrollees and instructional ly 
managed through RSVP by 13 instructors. The enrollment figures for the 
three terms in 1974-75 and Fall 1975-76 are shov/n in Table 1. As can 
be seen from Table 1, Open College courses are equally popular with men 
and women and approximately 50 percent of the enrollees are 30 years 
or older. 

The courses generally provide six to twelve RSVP surveys, each 
suirvey being made up of several questions. When the student sends his 
responses marked on a marksense card, he receives a RSVP letter (pre- 
constructed by the instructor) giving him prescriptions for the questions 

75 

62 



he missed and comiTi6n6; -^^-i. for those answered correctly. Full parti- 
cipation in the RSVP ifeStructional system would, then, mean that the 
student sent in all the RSVP surveys provided in a course and received 
corresponding number of RSVP letters. Participation in the system is 
not a requirement in open learning. However, when RSVP surveys are not 
turned in, the student receives a letter to that effect and is en- 
couraged to participate. 

Data for the Fall term, 1975-76 showed that two out of every three 
students utilized the RSVP system. 1 - participating group registered 
less number of course drops and greater number of course completions. 
(See Figure 1.) It should be mentioned that self-initiated participa- 
tion in the RSVP system was observed only In 15 percent of the students. 
The remaining 85 percent needed to be reminded. 

Correlational studies have repeatedly demonstrated the significant 
relationship of RSVP participation to course completion and performance 
in course examinations. These are summarized In Table 2 for two terms. 2 
Women and older persons tended to use the RSVP system to a significantly 
higher degree in most coi^rses. Correlation between RSVP utilization and 
exam score was consistently significant from term to term and for all 
courses. When effects of sex and age were held constant, the partial 
correlation between RSVP utilization and exam score was significant. 
Moreover, RSVP becomes a decisive winner when it cOi^':^ to students' 
evaluation of the courses offered in Open College. Term after term and 
course after course, students gave the highest rating to RSVP. (See 
Table 3.) 
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15% 

missed exani(s) 
but did not request 
for an I and got an 
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Figure 1 GRADE PERFORMANCE AND DROP RATE AS 
RELATED TO RSVP PARTICIPATION 
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TABLE 1 

TERM ENROLLMENT IN OPEN COLLEGE BY SEX AND AGE^ 
1974-75 and FALL 1975-76 





Fall 


Winter 


Sp/Sum. 


Fall 




1974-75 


1974-75 


1974-75 


1975-76 












Number of course offerings 


6 


8 


3 


8 


Number of course registrations 


1435 


3554 


686 


2219 


SEX 


% 


% 


% 


% 


Males 


45 


45 


59 


42 


Females 


54 


55 


40 


58 


AGE 










17 years or younger 


1 


3 


2 


1 


18 - 22 


20 


26 


22 


22 


23 - 29 


30 


25 


30 


28 


30 - 39 


27 


23 


27 


27 


40-49 


12 


12 


8 


13 


50 years or older 


11 


6 


12 


7 



1 Lack of information on sex or age is not shown in the table. 
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TABLE 2 
CORRELATION SUiARy 
WINTER 1974-75 and FALL 1975-76 



VARIABLES CONSIDERED 


ANT 


Ecy 


Ecy 


HSC 


HUM 


LAW 


LAW 


psy 




I.L\J 


ini 


in? 










911 


Winter ('74-'75) 


















Sexl & RSVP Utilization^ 


■ JLsi 


m 


20** 


17** 




11* 




11** 
• 10 


Age & RSVP Utilization 


.21** 




,28** 


.08** 




.16** 


.06 


.14** 


N 


328 


m 


512 


1238 


m 


306 


124 


518 


RSVP Utilization & exam score^ 


.42** 




.53** 


.46** 




.40** 


.32** 


.37** 


' N 


181 




334 


841 




190 


78 


274 


Fall ('75- '76) 


















Sex & RSVP Utilization 


.31** 




- 


.21** 


.21** 


^22** 


.32** 


.29** 


Age & RSVP Utilization 


.16* 


.14* 


• 


.11** 


.14** 


NS 


.31** 


NS 


N 


188 


250 


m 


557 


277 


195 


102 


314 


RSVP Utilization & exam score 


.49** 


.44** 




.51** 


.54** 


.45** 


NS 


.39** 


N 


109 


170 





404 


189 


120 


52 


182 



1 Sex: 1 = male; 2 = female 
"J J 2 Utilization = Number of RSVP surveys sent by student 
3 Exam score = t score on mid-tenn and end-term examinations 



* Significant at .05 level 
** Significant at .01 level 
NS Hon significant 
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TABLE 3 

STUDEMT EVALUATION OF COURSES OFFERED IN OPEN COLLEGE 
HMR 1974-75 



Statnent 


Percent of students reporting^ 
Strongly Agree Disagree Strongly 


Did not rate 
duetoieagre 




Agree Disaqree 


Participation 


iL^l^^i.^. T~ 111 * mm 

I believe I could have perforwd just as 






wll for this course without any telecasts. 


16 23 26 32 


3 


I believe I could have perforsed just as 






veil for this course vithout radio prograe. 


31 14 10 9 


19 


I believe I could have perfoned just as 






Nell for this course vithout RSVP surveys. 


♦ ' 5 15 73 


3 



g ^ 1 800 sfajdents participated in this evaluation. 
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Conclusion 

RSVP is a tool, nothing less and nothing more. Human minds con- 
ceive, create and invent tools. It is again the human minds who eval- 
uate, adopt and creatively use the tools. In the hands of an imagina- 
tive and skillful craftsman, a tool can not only accomplish what it is 
intended for, but go beyond intentions and discover creative, meaning- 
ful variations that might sometimes surprise even the originator. We 
see these possibilities for RSVP.^ In sharing with you our experiences 
with RSVP, we look forward to exciting dialogues between users of RSVP. 
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Footr ote 

1 A classification of computer programs for instniction is presented 
by Frank B. Baker in "Computer-Based, Instructional Management 
Systems: A First Look", Review of Educational Research . Vol. 41, 
February 1971, pp. 52-70. 

2 Courses represented are Cultural Anthropology (ANT 220), Ma>i and 
Environment (ECY 101 & 102), The Ascent of Man (HSC 101), Classic 
Theatre (HUM 204), Business Law (LAW 205 & 206) and Introduction to 
Psychology (PSY 211). 

3 Detailed information contained in comprehensive documents on RSVP is 
available by contacting tfie authors. 
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Program evaluation may be the least frequently engaged in 
activity at institutions of higher learning or maybe it's just 
the least frequently discussed. Certainly, it is typically not 
the kind of study one can easily get one's hands on. Requests 
I have made of locales where evaluation is "advertised" have not 
been fruitful. What I have been able to obtain in my quests for 
studies which might be somewhat helpful, if not replicable have 
been final reports of projects, or proposals to conduct 
evaluations, or suggested evaluation strategies and models. 
While these are all useful documents, the^ are not quite what I 
had in mind. We proceeded in our work at Florida International 
University, therefore, with what was felt to be an adequate 
theoretical basej but with ?idequate set of practical 

experienr.es to guide us: tha i, we knew a gre^ : deal about 
evaluation^ but we div-in't quite kno/ whet to expect. A gross 
oversimplification of our conclusions (ftfould have to include 
statements like; 

1) Now we know why reports of program evaluations a/e 
not shared, 

2) No one on this Eavth outside the uni ver^i *:y 
ccnd'xting th-^^T^ has tjrobably ever rer^vv^d a f i ll 
report of any evaluation, 

(maybe ift a morh serious vein) 

3) Pro».ran evaluation ''s vital for the maintenance of 
standards for any un'versity and, at the same time so 
probl emati/: 5 l.^av significant amounts of time and 
money must be ?*ilocdted for it^ completion. 
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But to begin back at the beginning: 

To guide our efforts, the following objectives were 

developed in consultation with the Academic n^^v.s for our program 

evaluation efforts • 

1) To orient decision making admin'^t iors to the 
advantages of the early developmc-,t and continued 
use of the program evaluation process.. 

2) ^0 select from the available literature an approach 
to evaluation which appeared to enhance program 
growth and development. 

3) To implement the evaluation model in a limited 
setting while carefullj^ monitoring its impact. 

4) To make necessary modifications in the model and 
then utilize it in a larger setting. 

5) To utilize the process university-vide for self-study 
purposes. 

The evaluation ^riodels which were examined may be conven- 
iently categorized as: 

Goal attainment models 

Judgmental models emphasizing Intrinsic criteria 

3. Judgmental models emphasizing extrinsic criteria 

4 . De c i s i 0 n - f a c i 1 i ta t i 0 r; mod e 1 s . 

In his book. Educational Evaluation . Popham identifies 
Ralph Tyler as the principal sponsor of the goal attai^Tient 
model wherein specific program goals are identified, translated 
into measureable objectives, after which pupils are tested to 
determine how well each objective has been attained. A program 
is evaluated then on the basis of how many of its objectives 
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have been attained. This is, of course, tl -.pproach utilized 
for the current National Assessment of Educational Progress. 

Judgmental models emphasizing intrinsic criteria or 
extrinsic criteria are categorized by their focur on the role 
of professional judgment. The most commonly utilized illustra- 
tion of the intrinsic variety of the judgmental model is the 
accreditation process wherein specialists from outside the 
school examine factors associated with the process of education 
with little or no attention being paid to the products of 
education. Thus, visiting teams count books in the library, 
faculty with PhD's and the number of elevators per student. 

Judgmental models emphasizing extrinsic criteria, general- 
ly associated with the writings of Michael Scriven and Robert 
Stake, place responsibility on the shoulders of the evaluator 
for, among other items: 

a. Distinguishing among formative and summativp 
evaluation needs, 

b. Assessing the quality of program ob ^•='r. ,:i > . 

c. Designing means of omparing effects o ' differ^ot 
programs. 

Throughout the abc^:/e b LtcruLion is paid to product in 
addition to, or often instead of, to process. 

The fourth class of evaluation models, dubbed ..^ocision- 
facilitation by Popham represented the focus we were looking 
for. Most advanced by Dan Stufflebt. > when he was at Ohio 
State, and by the UCLA Center for the liudy of Evaluation, 
directed by Marvin Alkin , these nodels stress most 
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fundamentally, the role of evaluation In enhancing the decision 
making process. Essentially, they provide a methodology which 
both aids In raising fundamental questions which need to be 
asked, and provides guidelines for resolving them. In fact, the 
decision-facilitation process uses elements of all three 
approaches previously discussed. The adaption of the 
Stufflebeam approach and of the CSE approach whic ; we utilized 
contained four stages, which appear as Figure 1 on page 11. 

Stage 1 - Needs Assessment ; in which the objectives for 
the program being evaluatecl^ are placed in the context of 
community needs. 

Questionr of concern Include ; 

Which objectives are these other programs meeting? 
Which objectives are not being met by any other 
programs? 

How may these objectives be prioritized in light 
or community needs? 
Stage 2 - Program Planning ; in which the particular 
collection of courses being offered are erA^^'ned in an 
endeavor to see their relationship to prog*":m objectives. 
Questions of concern Include : 

How have other programs differed from this one in 
array of courses, institutes, training programst 
etc., they have mounted to meet similar objectives? 
What evidence is there that this particular array of 
courses is more likely to reach program objectives 
than any other arrangement? 
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stage 3 - Program Implementation : in which difficulties 
encountered in the installation of any program component are 
identified (such as budgetary limitations, space limitations, 
etc . ) 

Questions of concern include : 

Has failure to reach objecti' .s, which is the concern 
of Stage 4, been due to a failure to properly 
implement key components of the program rather 
than a failure of the program itsalf? 
Can any mid-course corrections be made to increase the 
liklihocd that the progrJ=^m will be properly 
instal led? 

Stage 4 - Outcome Assessment : in whici. it is determined 
whether program objectives are being attained. 
Questions of concern include : 

Which objectives are not being attained? 
How does this relate to the needs assessment? 
Are changes needed in program planning or program 
impl emi^ii tation? 
With this strategy decided upon, meetings were conducted 
with administration and faculty of the School of Health and 
Social Services and the School of Hotel, Food and Travel, the 
two units selected fov* the pilot phase of the study. 

The four-stage evaluation began with a needs assessment 
for each program within the units. In this phase, interviews 
v^fere conducted with the program developers and all literature 
related to the development of the ;jrogram was read for the 
purpose of di'>cerning the particular needs the program was 
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established to meet. One outcome expected from this effort in 
addition to providing the foundation for later stages, was the 
appreciation on the part of the program planner of the 
changing nature of needs and the desirability of staying up-to- 
date relative to these changes. At the very least, it required 
planners to be aware of the rationale for the establishment of 
their program. 

In the second phase, program plf:inning , the rationale for 
the particular array of courses offered to students was explored 
in relationship to the needs the program was designed to meet 
and to other methodologies which could have been selected. 
A hoped for outcome here was the development of a plan to enable 
cost/benefit comparisons to be conducted across the various 
methodol ogies . 

In the third phase, program implementation , the mechanics 
of the operating pro^^^ram were examined to identify discrepancies 

(A 

between the program plan and reality. This phase is of 
particular importance in situations where failure to obtain 
objectives reflects a failure to properly install a program 
rather than a failure of the plan itself. This phase assisted 
the plann'^iTS in developing methods of "flagging" problems before 
they wert> likely to have too significant an impact. 

Fourthly, there was outcome assessment , in which data 
were collected from students participating in the program in 
order to obtain an indication of whether they felt the program's 
objectives were being attained. This information related 
directly back to the needs assessment phase where specific 
target objectives were identified- 
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The methodology chosen for outcome assessment was a 
telephone survey of graduates using specUiily trained 
interviewers. It was felt that this would significantly 
improve response rates. 

Figure 2 summarizes the approaches utilized in each 
stage of the process. Figure 3 is a copy of the outcome 
assessment questi'^nnaire. 

As to the results of this undertaking, it is important 
to point out first an? A that there now exists a climate 

for evaluation, a rec^ y to what it can accomplish. The 

same chairpersons who attempted to sabotage the process through 
long delays and seemingly endless requests for rewrites now 
turn to the Institutional Research Office when an outside source 
asks thciH '-or follow-up data on their graduates. We've come a 
long way in this regeird. It is recommended that the best way 
to prove one'r objectivity and sincerity when those come into 
question, and our experiences suggest they will, is to conduct 
one phase at a time, slowly, demonstrating^al 1 the while that 
you are professional in your activities and in the manner in 
which you handle information relating to programs you arc 
investigating. Evaluations are as much a part of the political 
process as they are of the research process and evaluators 
need tough skins as well as valid instruments. 

Secondly, after overcoming all of the above, our results 
suggest that it is possible for data from an objective evaluation 
to impact a program. The response rate to out telephone survey 
was astoundingly high. Actually, I'm not quite sure how to go 
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about assigning a percentage to this rate for a couple of 
reasons. The facts are these: 

Between 50% and 60% of all 1974 graduates were reached, 
that is, heard our voices and we heard theirs. Of that group, 
all but a very small handful (less than 5%) responded to our 
questionnaire- We have no reason to believe that the 40% we 
couldn't reach on the phone (unl isted numbers , moved, out-of- 
state numbers, etc.) differed in any systematic way from the 
group which responded. So, our response rate is either 60% 
or 95% I lean toward reporting the latter. 

The data showed program weaknesses where planners 
didn't know there were any. It even showed some strengths they 
hadn't previously known about. It revealed employment rates, 
salaries, and new needs graduates had themselves only recently 
identified. To be a bit more specific. In the School of Hotel, 
Food and Travel, the data led to a modification of the intern- 
ship program when graduates elaborated on it in the open ended 
questions . . 

In the Health area. Social Work and Health Sciences 
changed some attitudes regarding job placement; Nursing received 
outside input supporting its course objectives; Dietetics/ 
Nutrition received specific suggestions on ways , " .ngthen 
its offerings; Criminal Justice received information valuable 
for the development of its master's program; and data were 
obtaii^ed to substantiate a budget request for a field experi- 
ence program in sociology. This latter occurrence was 
particularly gratifying since the Office of Institutional 
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Research was both the recipient of the budget request and the 
planner of the study needed to objectively act upon that 
request. 

A third conclusion relates to the other aspects of the 
model. It is concluded that needs assessments are probaL,ly 
the most difficult (and least rewarding) undertakings for 
on-going programs. Ideall- the time to begin them is before a 
program is instituted so that the practice of gauging needs is 
built into the program design. In order to insure that on-going 
programs will engage in continuous needs assessment, it is 
necessary for the office doing the evaluation to supply ^the 
programs with simplified check lists and many guides to 
indicate where and how this kind of data may be obtained. 

Program heads are simply not oriented toward collecting 
and examining needs " assessment type data. The same is true of 
the program planning phase. Until that point when chief 
administrators require justification for the continuation of a 
given array of courses (and maybe that point is coming) there 
is little liklihood of program heads giving serious considera- 
tions to alternative sequences and methodologies. Program 
implementation results have been most informative and most 
useful. Program heads are accustomed to gathering this kind of 
information, albeit haphazardly, and rarely is anything done 
with this kind of data. Now we hav^ encouraged its systematic 
retrieval and analysis and consequently have enabled program 
heads to better summarize tlnn'r problems and needs. 

94 



79 



In conclusion, it is apparent that definite progress 
has been made toward accomplishing the objectives originally 
developed for this effort. And there is reason to be 
optimistic. 

The University is now considering a plan for the 
expansion of the process to the Schools of Business, Education 
and Technology, and to the College of Arts and Sciences. 
It is expected that the modified four-stage model will become 
fully "institutionalized" within the next five years, at which 
time, all year-end reports prepared by the Deans will include 
analyses of data relating to needs assessment, program planning 
program implementation, and outcome assessment. At that point 
in time, the University will have come a long way toward 
achieving its goal of data-based decision making. 
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STAGE 1 



STAGE 2 



STAGE 3 



STAGE i\ 



NEEDS ASSESSMENT 



IN WHICH THE OBJECTIVES FOR THE PROGRAM BEING 
EVALUATED ARE PLACED IN THE CONTEXT OF COMMUNITY 
NEEDS. 



PROGRAM PUNNING 



IN WHICH THE PARTICULAR COLLECTION OF COURSES BEING 
OFFERED ARE EXAMINED IN AN B^EMOH TO SEE THEIR 
RELATIONSHIP TO PROGRAM WECTIVES. 



PROGRAM IMPLEMENTATION 



IN WHICH DIFFICULTIES ENCOUNTERED IN THE INSTALLATION 
OF ANY PROGRAM COMPONENT ARE IDENTIFIED (SUCH AS 
BUDGETARY LIMITATIONS, SPACE LIMITATIONS, ETC.) 



OUTCOME ASSESSMENT 



IN WHICH IT IS DETERMINED WHETHER PROGRAM OBJECTIVES 
ARE BEING ATTAINED. 



FIGURE 1 
A FOUR-STAGE APPROACH TO 
PROGRAM EVALUATION, 
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STAGE 1 NEEDS ASSESSMENT 



Interviews with program developers 
Review of all related literature 
Review of Community needs. 

SESULI: Statement of objectives for the program. 



STAGE 2 PROGR.AM PLANNING 



Justification for particular array of courses 

Review of alternate approaches costs/benefit analysis. 

RESULT : Establishment of program to meet objectives. 



STAGE 3 PROGRAM IMPLEMENTATION 



On-site review 

Interview with key persons 

Review of budget/expenditures, 

SESiiLI: Decision as to whether program is liv :arget. 



STAGE ^ OUTCOME ASSESSMENT 



Research design to determine whether objectives . 

ARE being met 

Descriptive survey 

Experimental^ quasi -experimental study 

RESiiLI: Determination of which objectives have been 
achieved, 



FIGURE 2 

EVALUATION APPROACH FOR 
EACH STAGE OF THE PROGRAM 
EVALUATION PROCESS. 
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Telephone Survey 



Student's Program^ 
Tijne(fl) Called 



SECOND FOIXOW-OP 
OUTCOME ASSESSMENT 
CaSNERAL QUESTIONS/GRADUATES* 
SPRANG AND SUMMER QUARTERS, 1974 

1. Are you employed? 

V cs, full-tijne 

Y es, part-time, (please answer la below). 
N o, why not? (please skip to question 11) 

la. Are you employed less than full-time because full-time employxnent is not 
available or because you prefer part-tine employsient? 

F ull-time employment was not available. 
P refer part-time work. 

2. Are you employed in your career field? 

Yes 

N o, please explain 



3. Por whom do you work? 

4. What is your job title? ' 

5. Por how long have you held this position? 

6. Is this the same job you had before you got your degree from FIU? 

Yes No 

7. What is your current salary? 



8. What other positions have you held since you graduated from FIU? 



9. What are your feelings about your current job? (Would you rate it excellent, 
good, satisfactory or unsatisfactory) . 

I t is an excellent position 
I t is a good position 

It is satisfactory, but does not meet my expectations. 

Please indicate why: 
Salary too low 



_Poor working conditions 

"No advancement possibilities 



Other (Please specify) . 



10. Did your program at VIU provide you with the skills you need in your current 
position? 

Yes 

N o, pleaste explain (vrfiat skills were not provided) 



11. What was your major at the University? 

12. Do you feel your FIU program adequately prepared you for the near future? 

Yes ™ ' 

No, please explain. 



13. looking back, what do you consider *to have been the important strong points 
of you;, program? 



14. What were the weak points of your program?_ 
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OUTCOME ASSESSMENT 
QUESTIONNAIRE 



15. Do you have any suggestions for improving the programs in which you were 
enrolled at Flu? 



16* If you could add something to the curriculum, what would that be? 



17. rx> you feel the need for additional wtjrk at FIU? 

Yes 

No 

18. If so, what Xind of courses would be most helpful? : _ 

19. Are you presently enrolled in a graduate program at a university? 

Yes, which university 

N o ' Program: , 

20. If you are not enrolled, do you plan to enroll in a graduate program at a 
university in thie near future? 

Yes No 

21. In the space below, please add any other comments you care to make about your 
FIU experiences, now that you have been away for a couple of years. 



THANK YOU VERY MUCH! ! I ! 

•If Student was not available, who was interviewed?. 

Student's name 

Addr e s s_^ 

City, State " 



9a 

84 
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ABSTRACT 



The coroparison of the College-Level Examination Program (CLEP) and the 
Florida Twelfth Grade Test was designed to determine the number of underlying 
constructs common to both batteries, the extent of the redundancy between 
the batteries, and the comparability of the item characteristics for similar 
tests within the batteries. 

The data for the study were collected between June and December, 1974, 
and included scores for the students (252 Ss) who wrote the Florida Twelfth 
Grade -Test and all five of the CLEP General Examinations- A canonical variate 
analysis was conducted to establish the number of traits common to. both 
batteries. The Stewart and Love redundancy index was coznputed as a measure 
of the degree to which the batteries overlap;' Separate item analyses were 
conducted for the tests within the batteries • 

The results of the analyses indicated that the batteries shared four 
similar constructs representing a general ability factor and achievement in 
verbal, quantitative, and science skills. It was concluded that the tests 
measured similar processes, but tests within the CLEP battery were more 
difficult. If general achievement batteries are to be used for the placement 
of students at the college level, either battery would suffice assuming the 
differences in difficulty were taken into account. 
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REDUNDANCY AND DIFFICULTY: 

THE FLORIDA TWELFTH GRADE TEST 
AND 

THE COLLEGE-LEVEL EXAMINATION PROGRAM 

For twenty years, score?? from the Florida Twelfth Grade Test have 
been the wajor determinant for admission to the state universities of 
Florida* Over ninety percent of the graduating high school seniors take 
the test, and about half of those students continue at a pos t secondary . 
institution. 

There is considerable controversy over the appropriate use of 
achievea»ent test batteries. The controversy has intensified recently 
due to the expanded use of the Colltge-Level Examination Program (CLEP) • 
This program was designed in 1966 as a measure of achievement attained by 
persons outside an academic environment. The intent of the CLEP was to 
provide a mechanism for awarding credit for knowledge gained by persons 
with experience in the work force or elsewhere. In the State of Florida, 
there has been strong legislative pressure toward an accelerated collage 
degree program. As a result, five of the ten leading nationwide users of 
CLEP are educational institutions located in Florida. Most of the students 
seeking CLEP credit have been high school seniors or college freshmen. It 
appears that relatively few are older persons retuml^ng to the colleges. 
Therefore, the Florida Twelfth Grade Test (FTGT) determines eligibility for 
admission to the university system, and CLEP is used for placement of the same 
students. 
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Comparative Study 

Information on the statistical properties of the tests was compiled 
from reports by t^^a Testing Division of the Office of Instructional 
Resources and the College Entrance Examination Board. These statistics 
(Table 1) represent the properties of the norming population for the 
batteries, comparable data for the sample groups is presented in Table 1. 

Table L 

Coinparabl<e Statistical Properties 
of FTGT and CLE? General Examinations* 



Standard Error 



Subject 


Items 


Time 


Reliability 


of Measurement 


FIGT Aptitude 


100 


40 


.93 


4.18 


FTGT English 


85 


40 


.94 


3.71 


CLEP English 


95 


60 


.92 


31.00 


PTGT Social Science 


65 


40 


.92 


3.30 


CLEP Social Science 


90 


60 


.92 


28.00 


FTGT Natural Science 


60 


40 


.89 


3.09 


CLEP Natural Science 


85 


60 


.91 


30.00 


FTGT Mathematics 


60 


40 


.92 


2.97 


CLEP Mathematics 


80 


60 


.95 


24.00 


CLEP Humanities 


90 


60 


.91 


32.00 



*CLEP scores are based upon a standard score scale with a range 
from 200 tp 800. The mean is set at 500 and the standard deviation 
at 100. The FTGT scores are reported by percentile rank. The 
statistics ^^re derived from the raw scores. The difference in 
scoring accounts for the differences in the size of the standard 
error of measurement. Reliabilities for the Florida Twelfth Grade 
Test were estimated by the split half method and the Spearman- 
Brown correction formula. The Kuder-Richardson formula 20 was used 
to compute reliabilities for the CLEP battery. 

Purpose of the Study 

This study will address the problem of determining the comparability 

of the two test batteries in order to determine the degree to which the 

batteries may be used interchangeably. Comparability will be established 

by an examination of the underlying structure of the batteries > the degree 

of redundancy between the batteries, and the item characteristics of the 

batteries. 
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Comparative Study 

Astin (1972), in a review of the CLEP, stresaed the fieed f&t com- 
parative studies of the CLEP with established batteries. Given the variety 
of measurement ixeeds (e.g^ admissions 9 advanced placement , instifcitional 
self study and evaluation,), Astin Suggested studies which might determine 
the comparability of the CLEP with other similar test batteries 4n order 
to simplify testing procedures. 

It is important that comparative studies <?onslder the 6riginal 
purpose of the tests as well as their evolved use. Other factors also 
bear upon the comparability of tests. Similarities in content and diffi- 
culty level influence the predictive validity of the batteries. The test 
reliability and the effectiveness of a test in discriminating between 
students ;>f varying abilities must be determined. All of these consider- 
ations relate to the appropriate use of the tests. 

The Sample 

The Educational Testing Service was requested to release CLEP scores 
for the University of Florida students tested between June and December, 
1974. This record was matched against a computer tape record of University 
of Florida students who had posted Florida Twelfth Grade Test scores for 
1973. The selection process resulted in a sample of 252 jfe w ho had written 
all five CLEP General examinations and the Florida Twelfth Grade Test. Three 
parallel forms of the CLEP battery had been administered during the time 
period. It was decid^id to use the scores from all three forms to establish 
the correlations between the batteries. Item analyses were conducted using 
the scores from two of the CLEP forms which represented over two hundred 
subjects, 
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Cojnparatlve Study 

The Procedure 

A two part analysis was conducted^^ fizst to determine the canonical 
correlations between the underlying traits common to the batteries, and 
secondly, to explore the chs^racteristi(?s of th^ tests which affect the 
laagnitude of the correlations by item analyses of the tests. The Stewart \ 
and Love redundancy index will be reported as a measure of the overlap 
of total test variance between the batteries. 

The focus of the study was upon the correlations between the 
batteries, not upon developing equivalent scores • The present concern 
is to examine the similarities of the batteries as they are currently 
used. 

Results and Discussion 
The analyses have indicated that substantial similarity exists in 
the structure of the batteries. The four statistically significant canon- 
ical correlations can be interpreted to clarify these relationships 
(See Table 2). 

Table 2 

Results of Statistical Tests 

of Canonical Variates 



Canonical 
Roots 


Chi Squares 


Degrees of 
Freedom 


Probabilities 


.683 


282.666 


11 


0.01 


.512 


176.597 


9 


0.01 


.392 


122.325 • 


7 


0.01 


.130 


34.182 


5 


0.01 


.027 


6.843 


3 




.004 


1.164 


1 
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Comparative Study 

Loadings on the Varlates 

The first canonical relation has high loadings from each of the 
original tests (See Table 3). The fact that no specific test is pre- 
ponderant would imply that there is a common thread running through each 
battery. Undoubtedly, this thread is akin to a general ability "factor* 
Subsequent canonical relations are more closely Identified with achieve- 
ment in specific subject areas. The second set of canonical variates 
has high loadings for the CLEP mathematics test and the Florida mathe- 
matics and quantitative ability tasts. A verbal achievement variate 
emerges as the third canonical relation. High loadings for the English 
tests and moderate loadings from CLEP humanities and Florida Twelfth 
Grade verbal analogies combine to create this relation. The last sta- 
tistically significant set combines moderate loadings from the English 
and natural science tests. 
Redundancy Index 

The clearly defined patterns resulting from the canonical variate 
analysis confirmed the similarity of the batteries' structures. Another 
perspective on the comparability of the test batteries is attained by . 
examining the total redundancy between the test batteries (See Table 4). 
The canonical correlations represent the variance shared by the linear 
composites not by the original variables. The redundancy index is a 
measure of the total overlapping variance of the batteries. Within that 
context, it can be stated that there is substantial redundancy between 
the batteries. There is a forty-nine percent overlap in variance in 
either set of variates given the other set. As expected, the greatest 
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Table 3 

Matrix of Loadings on Canonical Variates 
for the CLEP and FTGT 



A-'Structure 
CLEP 


1 


2 


3 


4 


English 


.66 


-.25 


.60 


• .27 


Biological Science 


.69 


-.22 


-.39 


.32 


Physical Science 


.75 


-.02 


-.48 


.26 


Mathematics 


.76 


.65 


.01 


-.04 


Humanities 


.66 


-.39 


.29 


.03 


Social Science 


.79 


-.40 


-.16 


-.38 


B-Structure 
FTGT 


1 


2 


3 


4 


Verbal Aptitude 


.72 


-.16 


.33 


.03 


Quantitative Aptitude 


.67 


.57 


.09 


.03 


English 


.60 


-.14 


.67 


.24 


Social Science 


.78 


-.41 


-.24 


-.38 


Natural Science 


.72 


-.07 


-.49 


.49 


Mathematics 


.72 


.62 


-.02 


-.14 


Roots 


.68 


.51 


.39 


.13 


Canonical Correlations 


.83 


.72 


.63 


.36 



107 



92 



Comparative Study 



overlap Is contained in the general ability variate. Slightly less than 
three-fourths of the total redundancy is contained in the first variate 
for either set. The remaining redundancy decreases along with the variance 
extracted by the next three canonical variates. 

The fact that the redundancy index posits a sjnmnetric relationship 
between the batteries implies that either battery would predict scores 
on the other with equal efficiency. Predictions for the specific subject 
factors would be based upon a relatively small amount of variance. There- 
fore, a combination of the general ability component with the subject 
components would make a more reliable predictor. 

Table 4 



Components of the Stewart and Love Redundancy Measure 













Proportion 


Canonical 




Variance 




of Total 


Roots 




Squared 


Extracted 


Redundancy 


Redundancy 


CLEF as 


Criterion 


Set 








1 


.826 


.683 


.516 


.353 


.718 


2 


.716 


.512 


.140 


.071 


.146 


3 


.626 


.392 


.140 


.055 


.112 


4 


.360 


.129 


.064 


.008 


.017 


5 


.165 


.027 


,076 


.002 


.004 


6 


.068 


.004 


.061 


.000 


.000 


FTGT as 


Criterion 


Set 








1 


.826 


.683 


.493 


.337 


.694 


2 


.716 


.512 


.154 


.079 


.162 


3 


• .626 


.392 


.145 


.056 


.117 


4 


.360 


.129 


.077 


.010 


.020 


5 


.165 


.027 


.076 


.002 


.004 


6 


.068 


.004 


,052 


.000 


.000 



Note: Total variance extracted from CLEP = .997; R,. total redundancy 
for CLEP, g:..en FTGT = .491, Total variance extracted from FTGT = .997; 
R, total redundancy for FTGT, given CLEP = .486. 
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Comparative Study 
Item Analyses 

The statistics derived frcm ::he Item analyses were based upon raw, 
uncorrected scores. Means and standard deviations are reported below In 
Table 5 (See Table 5). 

Table 5 



Subject Test Means and Standard Deviations 
for the Sample Population 





Subject Test 


Standard 

Mean Deviation 


Number 
of Items 



English 



CLEP 


FC 


13 


48.65 


12.36 


95 


CLEP 


FC 


25 


50.84 


10.25 


95 


FTGT 


VRT 


63.21 


9.54 


85 


Social Science 












CLEP 


FC 


15 


40.30 


10.21 


90 


CLEP 


FC 


21 


41.20 


9.80 


90 


FTGT 


VRT 


43.81 


6.91 


65 


Mathematics 












CLEP 


FC 


13 


47.33 


10.66 


80 


CLEP 


FC 


25 


48.00 


10.21 


80 


FTGT 


VRT 


38.70 


7.19 


60 


Natural Science 












CLEP 


FC 


13 


39.58 


10.56 


85 


CLEP 


FC 


25 


43.35 


9.56 


85 


FTGT 


VRT 


38.70 


7.19 


60 



An understanding of the conditions which influence the magnitude of 
the correlations between sets of variables is achieved by reviewing the 
results of the item analyses. There are many sources of error in the 
measurement of true scores on tests which can distort correlations between 
similar tests. The size of correlation coefficients is affected by differences 
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in the administration of tests, the motivations of the students, guessing 
and scoring practices (Magnusson, 1967, pg. 101). Traditional measurement 
theory assumes that these sources of error are uncorrelated and normally 
distributed. These factors are likely to have attenuated the correlations 
reported in the study. But, these sources of error are integral parts of 
the testing programs. Therefore, comparisons of test reliabilities are 
made without correction for attenuation due to measurement error. 

Examining the internal consistency of the tests raises another issue. 
The internal consistency of the tests is a function of the length of the 
tests and the intercorrelations of the items. It is interesting to note 
that while the Florida Twelfth Grade Test has fewer items than the CLEP, 
reliabilities for the two batteries are not dissimilar (See Table 6). 
Adjusting the differences Ir; test length would minimize the differences 
which appear in the results. 

The average difficulties of the testing programs and their relative 
ability to discriminate among students is crucial to the comparison of 
the test batteries. Differences in the difficulty levels of similar tests 
reduce their correlation. It is apparent that the CLEP battery Is more 
difficult than the Florida Twelfth Grade Test (See Table 6). The mathe- 
matics tests are closest in difficulty level. The CLEP English tests and 
natural science tests are of average difficulty, and the CLEP social science 
test is somewhat more difficult. The Florida Twelfth Grade English and 
social science tests have a much lower average difficulty than their counter- 
parts on the CLEP battery. The Florida English test is in fact the easiest 
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achievement test on either battery for these students. 

The ability of an item to discriminate between low and high scoring- 
students is a crucial attribute. Very easy or very difficult items will not 
discriminate well between students. A comparison of the average discrimi- 
nating ability of the items in comparable tests indicate that both batteries 
tend to include items which have similar power to discriminate. While average 
discrimination le\ ^Is may be slightly higher for the CLEP batcery, differences 
between the two forms of the CLEP are as great between CLEP and the Florida 
Twelfth Grade Test (See Table 6). 

Table 6 

Statistical Properties 
of the CLEP anc^^ F TGT for the Sample 



Standard 





Form 


Code 


Error 


Difficulty 


Reliability 


Discrimination 


English 


CLEP 


FC 


13 


4.28 


50 


.88 


.34 


CLEP 


FC 


25 


4.22 


. 53 


.83 


.25 




FTGT 


VRT 


3.56 


74 


.86 


.27 


Social Science 


CLEP 


FC 


15 


4.08 


44 


.84 


.27 




CLEP 


FC 


21 


4.27 


*5 


.81 


.26 




FTGT 


VRT 


3.38 


66 


.76 


.26 


Mathematics 


CLEP 


FC 


13 


3.84 


58 


.87 


.33 




CLEP 


FC 


25 


3.82 


59 


.86 


.31 




FTGT 


VRT 


3.13 


64 


.81 


.29 


Natural Science 


CLEP 


FC 


13 


4.08 


46 


.85 


.31 




CLEP 


FC 


25 


4.04 


5P 


.82 


.28 




FTGT 


VRT 


3.32 


58 


.76 


.27 
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Comparative Study 

Concltislons 

The comparability of the test batteries was established in two 
stages. First, a canonical variate analysis was undertaken Cu find the 
underlying traits common to both batteries. It was concluded thaf, four 
traits were held in common. These traits were identified as a general 
ability factor, and mathematical, verbal, and science achievement factors. 
As part of the canonical analysis, the Stewart and Love redundancy index 
was computed. The index gives a measure of the proportion of total vari- 
ance that may be predicted by one test battery of the other battery. 
Thus, it is a measure of a total overlap in variance as well as a measure 
of symmetry between the batteries. There was forty-nine percent overlap 
in variance for either test, signifying a symmetric relationship between 
the batteries. Therefore, either battery would predict about half of the 
variance in the other. The strongest relationship was between the general 
ability factors followed by mathematics, verbal achievement and natural 
science factors. 

The symmetric relationship between the tests and the distribution 
of the variance within each battery is evidence of the similarity In content 
between the CLEP and the Florida Twelfth Grade Test. It must be recognized 
that substantial variance unique to each battery is also present. 

The correlations between the original tests as well as between the 
canonical variates were moderately, but not extremely high. 

The cdnditlons under which these batteries were compared would tend 
to maximize the differences between them. The CLEP includes a correction 
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for guessing in the scoring formula, the Florida Twelfth Grade Test does 
not. The batteries are of unequal length and no correction factor was used 
to estimate the results given tests of equal length. Finally, no correction 
for attenuation was used* In spite of this deliberate attempt to allow 
the differences between the batteries to become clear, the similarities are 
more apparent. The reliabilities for each battery are similar and differences 
may be attributed to differences in the length of the tests. The results 
of the item analyses indicate that the major difference between the two 
testing programs is in the difficulty level of the tests. The CLEP has more 
items with a high level of difficulty than does the Florida Twelfth Grade 
Test. The differences in the difficulty levels as Linn (1975) suggests, re- 
duce the correlations between tests which are in fact similar in content. 

The tests do measure similar content. However, the Florida Twelfth 
Grade Test is less difficult. Therefore, cutoff levels for the Florida 
battery for awarding advanced placement credit should be higher than cutoff 
on the CLEP. A predictive validity study has already been conducted for the 
Florida Twelfth Grade Test Honors Program. The results of that study have 
confirmed the predictive validity of scores of 475 or above on the Florida 
Twelfth Grade Test. The predictive validity of the CLEP has been established 
for students achieving scores above the 50th percentile of tne nationally 
normed sample. Other studies would be necessary to confirm the validity of 
using lower Florida Twelfth Grade scores for placement purposes. It. should 
be emphasized that the results only relate to University of Florida students. 
The results are not generalizable to other university student populations. 
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Do Not Equate 

An equating study resulting in score conversion tables could be 
undertaken. However, error would be expected in the equated scores. 
The correlations between comparable subject tests do not meet the usual 
standard of a .95 correlation between the tests. A more conservative approach 
would be to avoid actual score conversions. The Florida Twelfth Grade Test 
does identify high achieving students. If the cutoff for awarding credit 
is high on the Florida battery, the necessity for equated scorefii is obviated. 

Advanced placement through CLEP or the Florida Twelfth Grade Test 
has been granted in recognition of a general achievement in the field being 
tested. Either battery may be used for this purpose within the limits stated. 
However, the predictiva validity of a specific score for a specific course 
has not been established for either testing program. The tests are not a 
measure of knowledge of coiitent specific to an individual course. The 
philosophical argument of the legitimacy of granting credit using the 
mechanism of general achievement batteries remains. 
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Abstract 



Ihis is a five-year longitudinal study, the primary objective of which 
was to assess the educational validity or granting credit through the 
CLEP- For one design, five eommunity colleges selected experimental 
and control gtoups using a matGhed-group design. In a second design, 
students were selected who wrote the examinations and scored between 
the 25th and 49th percentile but did not receive credit and compared 
with others who did receive credit. Results indicate that students 
receiving CLEP credit compare favorably on selected dependent variables 
with like students not receiving CLEP credit. The data in this report 
represent student progress through two years of higher education. 
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Introduction 

The Task Force to Study Achievement of Students Granted Credit 
through the College Level Examination Program (CLEP) was charged by the 
State Articulation Coordinating Committee in April, 1973 with conducting 
a longitudinal study of the relationships between student achievement 
and the awarding of credit through the College Level Examination Program. 
Two major designs were developed by the Task Force to carry out this 
charge . 

Design I is a longitudinal study of students who enrolled originally 
at a community college and earned credit at the fiftieth percentile on 
college sophomore norms through the CLEP program and which follows their 
acadcjmic progress in the State University System. Design II is also a 
longitudinal study which follows the academic progress of community 
college students who wrote the CLEP examinations, but who received credit 
for scores at the twenty-fifth percentile on college sophomore norms. 
Background and Significance 

The College Level Examination Program, a national testing program 
used for awarding college credit through the process of standardized 
multiple-choice examinations, has been in operation for approximately 
ten years. The number of candidates writing the examinations has 
increased markedly during the past few years, and the Program has become 
quite popular with students throughout the country. In the State of 
Florida, approximately 11,831 students attempted the examinations and 
175,369 quarter hours of college credit were awarded during academic 
year 1973-74. The expanding use of the CLEP examinations for awarding 
college credit lias several important implications for the Florida higher 
educational system: a financial impact on state funding requirements, 
the potential savings to students in tuition fees, and the time gained 
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by students as their education is accelerated. 

An earlier study by (Losak & Lin, 1973) indicated that students 
earning CLEP credit progress at a faster rate than students of similar 
academic potential without CLEP credit, while at the same time not 
showing any significant decrease in later academic achievement. But 
critics of the CLEP program point out that courses represented by 
the General Examinations are not necessarily prerequisite for any 
particular advanced courses. Therefore, assessment of progress in 
advanced courses may not be a legitimate dependent variable on which 
to evaluate the effects of CLEP credit on student academic achieve- 
ment. The content validity of the CLEP program has also been ques- 
tioned, especially the mathematics examination. The College Entrance 
Examination Board, in cooperation with Educational Testing Service, 
is now in the process of establishing new norms for tne CLEP program 
with more rigorous content sampling as one of the objectives. 
Purpose - Design I 

The primary purpose of Design I was to compare the academic 
progress of two groups of students with similar Aptitude scores 
on the Florida Twelfth Grade Test: one group which received college 
credit through the College Level Examination Program (CLEP Group) 
and another group which did not attempt to receive credit through 
use of the CLEP examinations (Non-CLEP Group) . 
Procedures - Design I 

Five community colleges contributed data for Design I of the 
study. The number of students in the CLEP and Non-CLEP Groups from 
each college is presented in Table I. 
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TABLE I 

NUMBER OF STUDENTS IN SAMPLE BY COLLEGE 





CLEP 


Non-CLEP 




School 


Group 


Group . 


Totnl 


Daytona Beach Cocununlty College 


16 


16 


32 


Florida Junior College 


51 


51 


102 


Hlami-Dade Community College 


50 


50 


100 


St. Petersburg Junior College 


50 


50 


100 


Valencia Cocniunity College 


50 


50 


100 


Total 


217 


217 


A34 


Students were selected for 


the CLEP Group 


on the basis 


of having 



written and received credit for the College Level Examination Program 
during the calendar year 1972, with the further requirement that they 
had previously written the Florida Twelfth Grade Test (FTGT). In 
general, these students were f irst-time-in-college freshmen, who were 
born during the years 1953-55. 

The Non-CLEP Group was selected by matching students in the CLEP 
Group with other community college students of similar age and sex, 
and having comparable scores on the Aptitude section of the FTGT, but 
who did not attempt to receive credit through the CLEP examinations. 
Details of the matching procedure, which varied slightly among colleges 
are provided in the Appendix. Thus, it was assumed that the basic 
difference between the two groups for the purpose under study was 
whether the students had taken the CLEP examinations and received 
credit for their performance. 

The dependent variables of graduation rate, grade point average, 
and rate of academic progress through the college were chosen by the 
Task Force as factors reasonably obtainable from academic records. 
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Graduation was determined by counting only those students to whr,m each 
college officially awarded a degree. Grade point average was obtained 
from official transcripts and does not include CLEF credi . The grade 
point average for each group was calculated by adding grade point 
averages recorded for all students and obtaining a group mean. 

Rate of progress in college was calculated on the basis of months 
elapsed between enrollment and graduation, assuming four months for each 
major term, and two months for each short term. For example, if a 
student enrolled during the Fall Term, 1972 and graduated at the end of 
the Winter Term, 1974, he would have taken 20 months (fall, winter, 
spring, summer, fall, and winter) from enrollment to graduation using 
this definition. 
Results - Design I 

A summary of the characteristics of the CLEF and Non-CLEP Groups 
and an analysis of differences on selected variables between the two 
groups are presented in Table II. The mean number of credit hours 
awarded to the CLEF Group is virtually equivalent to a full semester 
of college credit. The number of credit hours attempted and earned 
through college enrollment by the CLEF Group was approximately four 
hours more than the hours for the Non-^CLEF Group. 

While it was assumed that matching of the two groups on the basis 
of FTGT Aptitude scores would assure reasonable equality on other vari- 
ables, such equality did not in fact occur. The CLEF Group had signi- 
ficantly higher FTGT scores in English, Social Science, Natural Science, 
and in Total score. Despite these differences in FTGT scores, both 
groups had Total scores which placed them in the top 20 percent of 
students who write the Florida Twelfth Grade Test. The CLEF Group 
grade point average of 3.07 was also significantly higher than the 
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£•83 grade point average earned by the Non-CLEP Group. 



TAiiLE II 

CHARACTERISTICS OF <!LE? AND NPN-CLF.P CROUPS 



VarlaSle 




CLEF Group 


Standard 


NON- 


-CLEP Group 
M 


C«-217) 


1 

t*"t^?t 1 

\ 


Ag€ ** 


216 


1 18.50 


1 2.01 


217 


» * 
1 16.29 • 


0.90 




Terns 

Fnrollcd 


217 


' 5.67 


' 2.37 


^217 


1 5.71 ^ 


2.82 




Hours 
Attempted 


217 


' 45.19 


1 16.67 


216 


1 41.79 , 


23.14 




Cours 
Zarocd 


217 




1 16.95 


216 


> 40.53 • 


23.64 




Hours 

CLE? 


217 


' . 14.07 


1 6.42 


0 


1 0.0 < 


0.0 




Total Hours 
Eara«d 


217 


f 60.00 


1 17.90 


217 


1 41.13 ' 


23.77 




Grade Point 
Averagt 


217 


' 3.07 


•1 0.59 


217 


1 2.83 


0.60 


3.65* 


r.T.G. ** 

Aptitude 


216 


' B4.ft5 


1 11.20 


216 


' 84.27 


1 11.39 


0.53 


F.T.G. 

English 


216 


i 82.33 


1 14.67 


216 


t 78.40 


\ 15.58 


2.70* 


r.T.c. 

Social Science 




I 

i 213 


' 84.06 


' 13.52 


214 


• 78.94 


1 15.46 


3.64* 


F.T.C. 

Katural Science 


212 


' 62.91 


* 13.22 


214 


1 76.65 


1 16.37 


4.03* 


r.T.G. 

Hath 


213 


' S2.20 


' 13. «4 


216 


• 60.53 


> 15.44 


I.IB 


F.T.G. 
Total 


213 


415.18 


' 49.75 


214 


1 400.15 


• 55.64 


2.54* 



♦Significant at ?^.05 
**Matching Variables 



Table III contains a breakdown of the kind of degrees awarded to 
members of each group through May, 1975. The CLEP Group had a gradua- 
tion rate of 61.29% which was twice as high as the graduation rate for 
the Non-CLEP Group of 31.34%. In comparing the length of time that 
graduates of each group attended college prior to receiving their degrees, 

108 

123 



it was found that the Non-CLEP Group graduates took 4.37 months (one 
full semester) longer to graduate than the members of the CLEP Group. 

TARIZ III 

DEGREES EARNED BY TH2 WlrTTER TERM, 1975 

CLEP Group (N-2I7) Noo-CLEP Group (H-2i7) 



Degree 


N 


z 


K 


Z 


AA Degree 


111 


56.68 


58 


26.73 


AS Degree 


10 


4.61 


10 - 


4.61 


Other Degree 


0 


0.00 


0 


o.oo' 


Planned Certificate 


0 


0.00 


0 


0.00 



Total Graduated 133 61.29 68 31.34 



Discussion - Design I 

Most of the students from' both the CLEP and Non-CLEP Groups are 
no longer enrolled at their original community college. A check of 
enrollment for the Fall Term, 1975 reveals that 20 (9.2%) of the 217 
CLEP Group members are currently enrolled,, while 27 (12.4%) of the 217 
Non-CLEP Group members are currently enrolled. It is, therefore, 
reasonable to assume that results as reported for each group are final 
euough for interpretations to be made with respect to their community 
college progress. 

Although both groups of students had FfGT scores V2ll above scores 
for the typical community college enrollee, for reasons unknown, the 
students in the Non-CLEP Group did not elect to seek college credit via 
examination. Their decision may reflect motivational or attitudinal 
differences not considered by the present study, or it may simply reflect 
a lack of awareness of the CLEP program. 

The results from Design I clearly indicate a higher and accelerated 
graduation rate as well as a higher grade point average for the students 
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in the CLEP Croup as compared with the Non-CLEP Group. These findings 
further corroborate those reported in the Losak and Lin study previously 
cited. It can be conservatively concluded that those students who 
received CLEP credit at the fiftieth percentile oa college sophomore 
norms were not placed at an educational disadvantage in comparison with 
students of similar academic potential when the comparison is made on 
rate of graauation and progress in the respective community colleges. 
The initial differences in FTGT scores earned by each group preclude 
a strong conclusion regarding the relationship between CLEP credit and 
grade point average. 

The initial charge of the Articulation Coordinating Committee 
requires continuing study of the students in both the CLEP and Non- 
CLEP Groups as they leave the community college and enroll in the 
State University System. The CLEP Task Force plans to monitor the 
academic achievement of students from both groups, and to this end has 
already engaged the cooperation of the University 'Registrars who will 
supply yearly transcripts beginning in June, 1976 for those students 
who have enrolled in their respective institutions. The Task Force will 
continue to report on the academic progress of students from each group 
during the next three years. 
Purpose - Desi^ II 

The purpose of Design II was to compare the academic progress of 
two groups of community college students, both of which wrote the CLEP 
examinations and earned similar scores, but one group was awarded credit 
(Credit Group) , while the other group received no credit for a similar 
performance (Non-Credit Group) . 
Procedures - Design II 

Miami-Dade Community College was the only community college having 
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a sufficient number of students who met the criterion to be included in 
the population studied. Therefore, the student sample in Design II 
consisted solely of students who matriculated at Miami-Dade. 

From July, 1970 to September, 1972, Miami-Dade Community College 
awarded credit for scores at or above the twenty-fifth percentile 
(college sophomore norms) on the ^^^EP examinations as recommended by the 
American Council on Education. Wi^> the implementation oc the Articu- 
lation Coordinating Committee Agreement regarding the CLEP program in 
September, 1972, the cut-off score for awarding credit was raised to 
the fiftieth percentile on~ college sophomore norms. 

The students selected for study in Design II were those who scored 
between the twenty-fifth and forty-ninth percentiles, with no more than 
one score between the fiftieth and sixtieth percentiles, and no score 
above the sixtieth percentile. 

The Credit Group consists of those students who wrote the CLEP 
examinations between July, 1970 and November, 1971 and were awarded 
credit for scores between the twenty-fifth and forty-ninth percentiles. 
Initially, there were 249 students in the Credit Group, but incomplete 
data reduced the final number of students to 191. 

The Non-Credit Groap consists of the 159 students who wrote the 
CLEP examinations during the time period from September, 1972 to June, 
1973 and who earned scores between the twenty-fifth and forty-ninth 
percentiles, but received no credit,- -since the cut-off score for awarding 
credit had been raised to the fiftieth percentile. 

The dependent variables determined by the Task Force for Design II 
were grade point average, rate of graduation, and rate of academic pro- 
gress through college. These were operationally defined in the same 

manner as for Design I. i o r» 

X ^ D 



Results - Design II 

A summary of the characteristics of the Credit and Non-Credit 
Groups and an analysis of differences on selected variables between the 
two groups are presented in Table I. 

.TABLE I 

CHARACTERISTICS OF CREDIT AND NON-CREDIT GROUPS 



I 



1 — ^ 

Variable 


N 


CREDIT Croup 
M 


fx- 191^ 
Standard 
Devlatloa 


Non-CREDIT Group (N-159) 
N M SD 


t-test 


Age 


1 AO 




1 7. 4 J 


155 


21.58 


, 6.90 




Hours 


IQI 

X7± 


XJ. JO 


' <i 7A 
O. /'f 


0 


' 0.0 


« 0.0 




Grade Point 
Average 


TCI 
±7 J. 


<c. // 


U. /o 


159 


' 2.94 


' . 0.72 


—2.12* 


F.T.G. 


112 


1 62.19 


1 on sn 


84 


1 62.88 


' 18.20 




F.T.G. 

EngHsh 


112 


' 59.68 


' 20.27 


34 


' 60.14 


^ 19.20 


-0.16 


F.T.G. 

Social Science 


87 


' 6A.72 


' 20.97 


75 


.65-12 


' 17.21 


-0.13 


F.T.G. 

Natural Science 


87 


' 64.67 


' 20.46 


75 ' 


61.44 


' 18.22 


1.05 


F.T.G. 

Kath 


110 


' 61.10 


' 24.46 


84 ' 


63.02 


J 18.78 


-0.60 


F.T.G. 

Total 


87 


' 326.29 
1 


J * 74.26 


75 ' 


317.27 


' 66.59 


0.81 



*Signif leant at P^.05 



The two groups did not differ significantly in their scores on the riorida 
Tvelfth Grade Test and thus are quite similar in terms of academic aptitude as 
measured through standardized testing. 
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There is approximately three years difference in mean age between 
the two groups which can be attributed to the earlier enrollment of the 
Credit Group at Miami-Dade. 

The Non-Credit Group grade point average of 2.94 was significantly 
higher than the 2.77 grade point average for the Credit Group. It should 
be noted that CLEP credits do not earn quality points and, therefore, 
neither increase nor decrease the grade point average. 

A breakdown of the degrees earned by graduates from each group 
is presented in Tables II and III. Through May, 1975, 56.5 percent of 
the Credit Group and 47.8 percent of the Non-Credit Group had graduated 
from Miami-Dade. 



TABLE III 

GRADUATION BY DEGREE FOR NON-CREDIT GROUP 



Degree 




Non-Credit Group (N-159) 
N % 


AA Degree 




54 


34.0 


AS Degree 




21 


13.2 


Other Degree 




0 


0.0 


Planned Certificate 




1 


0.6 


Total Graduated 




76 


47.8 


TABLE 


II 






GRADUATION BY DEGREE 


FOR 


CREDIT GROUP 




Degree ^ 




Credit Group 
N 


(N-iyi) 
z 


AA Degree 




89 


46.6 


AS Degree 




16 


8.4 


Other Degree 




3 


1.6 


Planned Certificate 




0 


0.0 


Total Graduated 




108 


56.5 
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Inhere were 13 students from the Credit Group (6.8%) and 20 students 
from the Non-Credit Group (12.6%) enrolled at Miami-Dade during the 
Fall Term, 1975. Comparisons between Tables II and III should not be 
made at this time since the Credit Group has been in college for a longer 
period of time. The results will be analyzed again in May, 1976 at which 
time nearly all of the students will have either graduated or no longer 
be enrolled, to determine any differences in the graduation rate for the 
two groups. Rate of progress in college will also be further analyzed 
at that time. A current analysis of time enrolled at Miami-Dade as 
measured by the number of months between enrollment and graduation indi- 
cates that the graduates of the Credit Group were enrolled for 25.61 
months, while the graduates of the Non-Credit Group were enrolled for 
31.76 months. 
Discussion - Design II 

The change in cut-off scores at Miami-Dade provided a unique oppor- 
tunity for a comparison of the Credit and Non-Credit Groups which elimi- 
nated the factor of motivation related to applying for and taking exami- 
nations. It is assumed that the students in each* group were equally 
motivated since they sought credit through the CLEP examinations. The 
fact that both groups scored at similar levels on the CLEP examinations 
is a further argument against initial differences between the two groups. 

The preliminary analysis of the data suggests that students awarded 
CLEP credit make more rapid progress * through the community college and 
graduate at a higher rate, but with a grade point average which is 
significantly lower than the grade point average earned by students who 
did not receive credit for the CLEP examinations. Further interpretation 
of the data and their implications for the Florida higher education 
system should not be undertaken until the final report of Design II of 
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the Task Force is presented. 

In addition to the brief Appendix which follows, more complete infor- 
mation regarding background and development for each study, including 
forms developed by the Task Force, and Minutes of Task Force meetings 
may be obtained upon request to the Chairman of the Task Force. 
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ABSTRACT 



MID-POINT TESTING PROGRAM: 
BEAUTY OR BEAST? 

Limitations on enrollment in upper division units within the State 
University System of Florida may well give iiiq)etus to the development of 
a mid-point (end of sophomore year, beginning of junior year) testing program 
for all baccalaureate students. This paper examines the purpose (s) such a 
program might serve: screening, certification, placement, accotintability 
or degree by examination. 
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MID-POINT TESTING PROGRAMS 
BEAUTY OR BEAST? 



During the past two years, decreases In state revenued and increasing 
costs in materials and services have put a severe financial strain on the in- 
stitutions of higher education in the State of Florida. As the result of 
decreasing resources, limitations on enrollment are being considered in some 
institutions, and have become realities in others. The universities' open 
door to all who hold associate degrees from state community colleges may well 
be closing to some. 

If an institution must begin limiting enrollment to all upper division 
programs^ then careful and thoughtful consideration must be given to how 
students are to be selected. One alternative would be to espouse the demo- 
cratic procedures of selection by lottery, an alternative championed by some 
in the legislature during the 1976 session. 

Most of us, having been subjected to a long series of admission require- 
ments, may find that this procedure,, although attractive from a populist point 
of view, would be difficult to defend to our faculties and students. A second 
alternative which has been proposed and discussed in several circles is the 
establishment of a mid-point (end of the sophomore year-beginning of the 
Junior year) testing program. It is to this proposal that I would like to 
address my comments by examining some of the Issues involved. 

It would seem on the surface that the major issue Involved in establish- 
ing such a testing program would be the selection or development of the instru- 
ments to be used* However, before test selection or development can be undertaken 
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the purposes for such an examination should be clearly stipulated. A realm 

of possibilities exist in terms of purpose. 

Screening 

The selection of students by administering aptitude and/or achievement 
measures could be the major purpose of such a program. The question becomes, 
screening for what? The Btate University System now uses standardized admission 
tests for incoming freshman which provide measures identified as aptitude, ability, 
or educational development (Florida Twelfth Grade Test, American College Testing, 
Scholastic Aptitude Test). Whatever the name, each of these tests reveal a 
large general intelligence component with ancillary verbal and quantitative measures 
when submitted to factor analysis. These global measures have been used to sort 
out from the general secondary school population those who score high on such 
tests. 

However, it is not reasonable to assume that screening for an elitist 
population should be the purpose of a mid-point examination. One could propose 
the reverse approach be made, that the screening be done to eliminate only the 
weakest of students who score very low on such measures. Either purpose, to 
skim off the top or eliminate the bottom of the range for a test which measures 
basic intelligence factors would deny the very purpose of the associate degree. 
As our own Council of Deans concluded, a mid-point examination should measure 
the achievement of students who have concluded a general education program, 
not their general aptitude. Attractive as this stance may be. It presents serious 
problems for those of us who are responsible for providing input into the decisions 
for such a testing program. Basically, the Council of Deans' decision makes two 
assumptions. First, that there are easily identifiable objectives and content 
which are common to all general education programs throughout the state institutions 
of higher education which could be used to develop specifications for achievement 
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measures. Secondly, that general education programs are a necessary prerequisite 
for success in upper division liberal arts and professional programs. Neither 
of these assumptions are valid. There is no common set of precisely described 
goals for general education, nor has anyone defined a body of information or a 
common set of skills tawght in general education courses which are necessary 
for success in upper division curricula^ There may indeed be a better argument 
for discreet criteria for admission to specific programs; that is, the qualifi- 
cations and skills for a student entering an electrical engineering program may 
be distinctly different from those for a student training to be a professional 
artist and still different for a student planning to become a teacher of English 
in the secondary schools. 

This leads back full circle to the original question, screening for what? 

Certification 

A different approach which may have considerable merit is the establish- 
ment of a testing program or programs that would certify that a student, upon 
completion of an associate degree, has met certain criteria or has demonstrated 
a set of competencies defined by the particular program he has completed. Rather 
than a beginning of the junior year testing program administered by the state 
universities, it would become an end of the sophomore year program administered 
by commmnity colleges and lower division units within the universities. The 
advantage of such an approach would be that the degree granting institution 
would assume the responsibility for certifying its own graduates in relation to 
its own instructional , programs. However, unless the competencies were defined 
in terms of something like basic literacy, a certification that the student 
could meet established standards of reading and writing skills, the plethora of 
tests to be developed could well become prohibitive in cost and burdensome to 

admission officers. -i o r 
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Placement 

On our own campus there is growing support for a mid-point testing program 
which has as its major purpose a guidance function for the placement of students 
in appropriate courses and ultimately in appropriate curricula. As the first 
step in the development of such a program^ the College of Arts and Sciences, 
which admits approximately four thousand students each year, has made plans to 
launch a pilot study this fall. A common requisite to all Arts and Science 
programs is the ability to write standard, that is conventional, edited English* 
There is increasing concern on the part of factilty that many students, both 
native and transfer population, lack this ability. In fact, there seems to be 
fairly uniform agreement that many students at the University of Florida do not 
write well. However, there is little systematic empirical data which supports 
this contention or describes the magnitude of the problem. In an attempt to 
develop a uniform method of assessing writing skills, two testing programs — 
the Regents Testing Program , an essay examination used by the State University 
System of Georgia and the Test of Standard Written English developed by Educational 
Testing Service for the College Entrance Examination Board— will be used in a 
two part study to establish the usefulness of these tests for the assessment 
of our students* writing skills. 

There will be no attempt at this time to use one or both of these tests 
to exclude students. Rather, they will be used to place students in appropriate 
courses or remedial laboratories if the data indicates such action may be benefi- 
cial to the student. 
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The ultimate goal of this study after a three year period will be to 
answer the following questions: 

(1) Do one or both of the tests parallel writing instruction of 
the English Department? 

(2) Do one or both tests predict success for students In various 
programs? 

(3) Can a cut-off score or level of performance be established below 
which a student will experience difficulty In performing adequately 
In a specific program? 

If affirmative answers are found for these questions, then the evidence 
may Indicate that an effective Instrument has been Identified which could 
be then used as a screening device for any upper division academic program 
which places a high emjphasls on writing skills. The major caution would be 
to use neither of these Instruments as a screening device before the three year 
study la completed. 

There are other existing Instruments which tap skills and knowledges 
Identified as major prerequisites for specified currlciila which could be studied 
through this approach. However, without a careful study of the predictive 
ability of such Instruments, their value cannot be determined. 
Accountability 

Yet another purpose T^r which a mld-polnt testing program could be used 
Is accountability. One of the grave dangers In designing a screening and/or 
placement program which may have excellent predictive qualities and thus be 
highly useful at the upper dlvlclon levels of the universities Is that It will 
automatically be assumed by some that It can also be a useful Instrument for 
determining the effectiveness of the various community colleges and lower division 
instructional programs. Those of us who have an understanding of the fallacies 
in such an assumption and who have seen some of the invalid comparisons which were 
made from the results of the Florida Twelfth Grade Testing program, can anticipate 
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that a mid-point screening device could become a weapon tuimed upon the community 
colleges and lower division programs. Program monitoring and evaluation in terms 
of student achievement are important activities, vital input to educational 
decision making. However, xf a mid-point testing program is to be put to this 
use, that purpose she aid be clearly determined at the onset and the testing 
programs so designed. The accent must be on the plural, for program evaluation 
demands measurement of the specific components of the program, not global measures 
which will be more influenced by the characteristics of the students as they 
enter the program than by the impact the program has on their learning. 
Degree by Examination 

Two studies of programs by which credit-examination is awarded have been 
presented in these proceedings. It does not take a very large intuitive leap 
to assign yet another purpose to a mid-point testing program, an associate degree 
by examination. If a testing program is designed to measure the achievement of 
students who have completed general education and pre-prof essional curricula then 
it would not be unreasonable for in-coning students to challenge the tests and 
be awarded a degree upon the basis of their successful performance. Needless to 
say, this idea might be extremely atractive to some of our cost conscious politi- 
cal leaders, but I can state with assurance it would meet with even less enthu- 
siasm with my own faculty than the College-Level Examination Program ever did. 
Conclusions 

I have attempted to share with you some of my thinking concerning one of 
the issues surrounding the establishment of a mid-point testing program. Basically, 
it revolves around the purposes such a program or programs would serve. I have 
not suggested the resolution of the identified ir ue; indeed it would be presumptuous 
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for me to do so. Rathe:, I urge the community of institutional researchers to 
address t^^se Issues on their own campuses and begin developing some pilot 
programs to provide empirical Information which can be used for program develop- 
ment purposes. 

A well designed program, ba*aed on well articulated purposes and strongly 
supported by faculty can provide useful Information for effective decision 
making. A h-TStlly conceived program used for a multiplicity of purposes for 
which It was not designed may place us In the position of having grabbed a 
"tiger by the tall". I firmly believe as Institutional researches, we should 
work toward dealing with this problem. If we don't, someone will do it for us. 
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ABSTRACT 

This study was undertaken to evaluate an orientation program that 
purported to reduce the level of underachievement normally associated with 
first-quarter students and to reduce the attrition rate of new students 
during the first year of college. The basic issue a^^iressed by this paper 
is "Can an orientation program be developed which will decrease^ if not 
eliminate, the high attrition rate that occurs during the first year of 
higher education." This study contrasted attrition rates and achievement 
levels for students who participated in a special orientation program during 
the summer of 1974 and a control group who participated in the regular 
program. Tlxe special orientation program provided students the opportunity 
to participate in several activities designed to improve their decision- 
making skills and to decrease their level of anxiety. The control group 
participated in a regular orientation program that was designed to inform 
students of specific institutional requirements and familiarize them with 
the university service programs and facilities. 

The research findings of this study appear to indicate that the overall 
achievement level during the first two quarters of college work can be 
improved and that the attrition rate can be reduced by providing appropriate 
orientation experiences. Furthermore, the study seems to indicate that the 
academic capable student is more likely to participate in experiences 
similar to Mr. Ferrell's orientation program. 

Since the findings of this study were based on only two quarters of data 
with the possibility of other variables such as motivation accounting for the 
differences in grade point average and attrition rate, further analysis must 
be completed before difinitive recommendations can be made. 
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THE EFFECTiraTESS OF M INCOMING 
STUDENT ORIENTATION PROGRAM 

Introduction 

For several centuries institutional personnel have expressed concern over 
the high attrition rate of students who enter higher educational institutions. 
A high attrition rate represents^ a serious problem to both the student and 
the university. It is a financial burden for the student and requires the 
utilization of inore resources to maintain a specified enrollment level. 
Attrition also creates an emotional crisis for the student who experiences 
failure. When students who have the potential to succeed are the victims 
of attrition, the potential human resources of society are underutilized. 
Since institutional programs can be developed which wdLll decrease the rate 
of attrition while maintaining quality standards of academic excellence, 
these societal losses can be prevented. 

Although a considerable amount of research has been completed which 
indicates the attrition rate Is very high during the first year and that 
attrition is related to the student's level of integration in both the social 
and academic systems of an institution, a minimal effort has been made to 
determine what can be done to control this phenomenon. A longitudinal study 
completed by Trent and Medsker indicates that in all probability the level 
of persistence can be improved if the level of discontent and insecurity can 
be reduced (Trent & Medsker, 1968). 

The basic issue being addressed in this paper is can an orientation 
program be developed which will decrease, if not eliminate, the high attrition 
rate that occurs during the first year of higher education. The purpose 
of this study was to evaluate an orientation program which purported: 

1, to reduce the level of underachievement normally associated with 
first-quarter students; 

2. to reduce the attrition rate of new students during the first year 
of college. 142 
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Overview 

A brief review of the literature on attrition revealed some basic 
concerns associated with the topic. They are as follows: 

1. The high attrition rate contributes to the underutilization of 
valuable human resources as well as creating a financial burden 
for those who withdraw; 

2. The institutional cost associated with the recruitment and admission 
effort to replace the first year nonpersisters create a financial 
burden on the institution. 

Huber reported that for students who entered as freshmen, the graduation 
rate falls between 30-50 percent in public universities. Furthermore, attri- 
tion ranes have remained approximately constant over the past three decades 
(Huber. 1971) even though student mix has changed drastically. Most 
institutional personnel consider a high attrition rate to be a natural 
phenomenon. However, when the concern is addressed, students are considered 
to have the shortcomings and are blamed for the problem. The most common 
ns cited for high attrition rates are that students have low ability, 
lack ambition, dislike responsibility, can not read, are poorly motivated, 
and lack direction (Huber. 1971; Trent & Medsker. 1968). 

Several authors cite using student services as a possible solution to 
the attrition problem. Trent and Medsker reported many of their subjects 
indicated that the need for direction was essential. Likewise, they indicated 
the goal of counseling should be to increase self-responsibility and to. increase 
the level of decision-making maturity of students (Trent & Medsker. 1968). 
Cochran noted that most successful programs demonstrate a concern for what 
happens to the student beyond the admission stage. One such approach included 
prograrns which utilized the process of peer aid in the organization of academic 
and counseling service programs (Cochran, 1975). Huber contended if one were 
to determine individual needs of the student at the point of entry one could 
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predict likelihood of self-fulf llliaent (Huber, 1971). However, this would 
necessitate that programs and priorities be established by all of the 
institutions within a state thus allowing for the alignment of student needs 
and institutional programs. 

Regardless of the potential merit of new and innovative programs^ the 
conflicting pressures related to program effectiveness and budget constraints 
necessitates that programs be evaluated to determine both their usefulness and 
cost effectiveness • ^ 

Special Orientation Program 

A special orientation program was developed by Mr, Jlmmie Ferrell, 
Director of Student Organizations, and implemented during the summer of 1974* 
This program provided students the opportunity to participate in several 
activities designed to improve their decision -making skills and to decrease 
their level of anxiety. These activities were as follows: 

1. Each student was associated with a student/faculty facilitator team* 
This activity enabled the student to become oriented to the University 
in a very personal manner with eiz^)hasis on "accent on the individual/' 

2. Each student participated in development exercises which created an 
awareness of the dynamics of interpersonal relationships such as 
leadership skills » listening communication skills, and decision- 
making skills^ 

3^. Each student participated in exercises designed to develop specific 
short and long-range educational goals with emphasis on career 
planning* This activity was designed to increase the probability of 
success at the University; 

4. Each student participated in an orientation program to familiarize 
him/her with the student service and academic service programs, . 
including Cooperative Education, Developmental Center, Financial Aid, 
ROTC, and Library Service; 
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5. Each student participated in activities which were designed to stimu- 
late both their academic and extracurricular involvement; 

6. Each student participated in a personalized academic advisement and 
registration process in an attempt to decrease the level of appre- 
hension commonly associated with the new entering fresfamaxi* 

Population 

The population for this study Included 819 first-time-in-college students 
who attended Florida Technological University during Fall Quarter, 1974. The 
population was stratified into two groups. The first group (special group) 
was composed of those students who applied prior to June I and chose to attend 
the special orientation program which was held during the summer term. The 
second group (control group) participated in a regular orientation program that 
was designed to inform students of specific institutional requirements and 
familiarize them with the University service programs and facilities. 

Statistical Method 

The statistical techniques utilized included an ANOVA, Covariance analysis 
and Chi-square analysis. The null hypotheses of no difference between the 
overall grade point averages for the two populations by Florida Twelfth Grade 
categox^r were tested with an analysis variance design and followed up with a 
Covariance analysis such that variability in achievement attributable to dif- 
ferences in admission test scores were eliminated. The null hypotheses of 
independence were tested with a Chi-square analysis. For this analysis each 
group was stratified by three Florida Twelfth Grade score categories. 

Research Finding 

The first-time-in-college population was distributed between the special 
group and the control group. Table 1 provides a brief demographic profile of 
the two groups. These data revealed that the average score on each measure 
was greater for the special group. J ^ ^5 
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TABLE 1 

Denijographic Profile 
By Group 



Category 


Special Group 


Control Group 


Total 


391.5 


376.9 


PSY 


74.09 


53.77 


GPA 


2.53 


2.33 


N 


389 


430 



The null hypothesis of no difference between population mean overall 
grade point averages for the first two quarters was tested with an ANOVA design. 
Table 2 provides a brief summary of these findings. Based on the results, the 
null hypothesis of no difference was rejected, at the .002 level of significance, 

TABLE 2 I 
Summary of "ANOVA" Findings 



Source of Variation 


Sum of Squares 


df 


Mean Square 


F 


Significance of F 




8.063 


1 


. 8.063 


10.67 


.002 


Residual 


617.326 


817 


.756 






TOTAL 


625.389 


818 


.765 







These data revealed a definite relationship betroen a student participating in 
the special orientation program and their overall ^rade point average achieve-- 
ment at the University, The 95 percent confidence Intervals for the overall 
grade point average are as follows: 

Control Group - 2.24<.^fGPA< 2.42 

Special Orientation Group - 2.46<-^GPA <2.61 



146 

131 



These data evidence that the confidence intervals do not overlap and the 
expected mean GPA difference is .2 of a point on the 4 point scale. 

Since the deiaographic characteristics of the two samples were not similar, 
a covariant analysis was conducted to eliminate the variability In grade point 
average attributable to total Florida twelfth grade score. Table 3 provides 
a summairy of the covariant MOVA findings. Although the total Florida twelfth 
grade test score was a significant contributor, these data revealed that the 
special orientation program appeared to have a positive effect on academic 
achievement. Based on these findings the null hypothesis of no difference was 
rejected at the .033 level of significance. 



Table 3 

Summary of "Covariant ANOVA" Findings 



Source of Variation 


Sum of Squares 


df 


Mean Square 


F 


Significance of F 


Covariant 












Fla. 12th Grade 


32.231 


1 


32.231 


44.58 


.001 


Mala Effects 












ID 


3,228 


1 


3.228 


24.524 


.033 


Explained 


35.459 


2 


17.729 


24.524 


.001 


Residual 


589.930 


816 


.723 ■ 






TOTAL 


625.389 


818 


.765 







A Chi-square analysis was used to determine if attrition was dependent on 
the type of orientation program. Table 4 provides a summary distribution oC 
the number and percentage of students who dropped and continued by group. A 
contingency table analysis of these data evidenced a coiiputed Chi-square of 
25.91. With one degree of freedom, the null hypothesis of independence was 
rejected at the .001 level of significance. These findings revealed that 10.5 
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TABLE 4 

Number and Percent of Students Who 
Dropped or Continued by Group 



Attendance 


Special 


Control 


Total 


Category 


Group 


Group 


Dropped 




• 




aUmuG^ 


/• 1 


105 


146 


XvUW £^c£^L.cUL 


Oil 1 7 
4LO9 1/0 


71,9Z 


17. 8Z 










Continued 












32 J 


f ^ 1 

673 


Row Percent 




Aft 




Column Percent 


89.5% 


75.6% 




Total 








Number 


389 


430 


819 


Percent 


47,5% 


52.5% 


lOOZ 



X a 25.91 Significance Level « 0,0001 



percent of the special orientation group dropped out of college as compared 
^ to 24,4 percent of the control group. As In the ANOVA design, steps were 
taken to determine if the demographic characteristics of the two groups were 
accountable for the difference In attrition rates. To eliminate the external 
Influence of the Florida twelfth grade score, the groups were stratified into 
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the following subgroups: 

Lov Subgroup - FTG < 350 

Medium Subgroup - 350^ FTG ^415 

High Subgroup - FTG >. 416 
A Chl-^quare aualysio was completed for each subgroup. Table 5, Table 6, 
and Table 7 provide a distribution of the number and percent of students 
who dropped or continued by subgroup* 

TABLE 5 

Number and Percent of Students 
Who Dropped or Continued 

Low Subgroup 



Attendance 
Category 


Special 
Group 


Control 
Group 


Total 


Dropped 








numoer 


lO 


DO 


70 


Row Percent 


22.2% 


77.8% 


26.3% 


Coltimn Percent 


15.4% 


32.9% 




Continued 








Number 


88 


114 


202 


Row Percent 


43.6% 


56.4% 


73.7% 


Column Percent 


84.6% 


67.1% 




Total 








Number 


104 


170 


274 


Percent 


38% 


62% 


100% 
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TABLE 8 

Number and Percent of Students 
Who Dropped or Continued 
Medium Subgroup 



Attendance 


Special 


Control 


Total 


Category 


Group 


Group 


Dropped 








Number 


15. 


35 


50 


Row Percent 


30. IZ 


70.0% 


18.8% 


Colum Percent 


11. 6Z 


25.5% 




Continued 








Number 


114 


102 


216 


Row Percent 


52. 8Z 


47.2% 


81.2% 


Column Percent 


88 .4% 


74.5% 




Total 








Number 


129 


137 


266 


Percent 


48.5% 


51.5% 


100% 
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TABLE 7 

Number and Percent of Students 
Who Dropped or Continued 
High Subgroup 



Attendance 
Category 


Special 
Group 


Control 
Group 


Total 


Dropped 








Number 


10 


14 


24 


Row Percent 


42. 7Z 


58.3% 


8.6% 


Column Percent 


6.4% 


11.4% 




Continued 








Number 


146 


109 


255 


Row Percent 


57.3% 


42.7% 


91.4% 


ColtiEza Percent 


93.6% 


88.6% 




Total 








Number 


156 


123 


279 


Percent 


55.9% 


44.1% 


iao% 



Table 8 presents a suinznary of the X analysis by subgroup. Based on 
these findings the null hypotheses of independence were rejected the 
•002 and .006 level of significance for the low subgroup and medium subgroup 
respectively. These data revealed that attrition was dependent on the type 
of orientation program for the low subgroup and the medium subgroup. Based 
on these findings it sepms apparent that the special orientation program 
was successful in reducing the overall attrition rate for two of the three 
subgroups. It appears that only the high subgroup did not benefit from the 
special orientation progrrim. 
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TABLE 8 



Chl-Square Analysis Suznaary 
Data by Subgroup 



Subgroup 


N 


x2 


d£ 


Significance 






Level 


Low 


274 


9.38 


1 


.002 


Medlua 


266 


7.45 


1 


.006 


High 


279 


1.58 


1 


.21 



Summairy 

These findings appear to Indicate that the overall achievement during 
the first two quarters can be licproved and that the attrition rate can be 
reduced by providing an orientation experience similar to Mr. Fcrrell's 
program* This study provides evidence which seems to Indicate that the 
academically capable student is more likely to participate in experiences 
similar to Mr. Ferrell's program when they are arranged on a voluntary basis. 
Accordingly, these findings indicate that the experience of Mr. Ferrell's 
program was Influential in bringing about a positive change in the areas of 
achievement and attrition. However, it is important to note that one can 
not assuxse the objectives of the special orientation program were realized 
since a formal evaluation was not imdertaken to determine if either the 
student's decision-making skills were enhanced or their level of anxiety was 
lowered after participating in the program. 

Since these findings were based on only two quarters of data with the 
possibility of other variables such as motivation accounting for the dif- 
ferences in grade point average and attrition rate, further analysis must be 
completed before definitive recommendations can be made. 
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THE COST-INCOME COMPONENT 



OF PROGRAiyi EVALUi. ON 
Norris Miner 
Seminole Community College 

A very wise lady from the hill country of southern 
Indiana has developed a philosophy over the many years of her 
life that allows her to live happily with her meager lot. As 
she will tell the restless young who will pause to listen; 
"Tell me what thee wants, and I will tell thee how to live 
without it.*' 

The Florida Legislature has been using this philosophy 
with Florida public post-secondary education for several years. 
Some programs which post-secondary education wants may be those 
things we will have to learn to live without. Painful evalua- 
tions are necessary to learn what programs and how much. 

Although program evaluation is a theoretical component 
of every educational planning and budgeting cycle, it is not 
necessarily a realistic or meaningful endeavor during times 
of reasonable funding. Program decisions too often appear 
to be made on a changing set of valued with little, if any, 
cost and/or income information. 

The^ value component is, of course, the major component 
in any program decision. However, values have often been ill- 
defined. V'^gue terms such as, "community commitment'*, "education- 
ally sound," "meets state or national needs", "adds presticre", 
and "supports other programs" have become values on which programs 
have been justified. Documentation of values such as those just 
listed has not always been produced or if it has, the documen- 
tation has been as vague as the terminology. Under the present 
stress of budget restrictions, values need to bG defined in 
more precise terms. Program retention or expansion now V9ry 
often encroaches upon other programs of the institution in a very 
real and restrictive manner. Values must be examined not only 
in the program under consideration but also in any other program 
affected. Thus, value definition v/ill have to be precisely 
stated in orcter to allow for the necei^sary give and take. 

The second component of program evaluation is an examination 
of the cost to income relationship of the program. This cost to 
income relationship becomes a value to add to the mix. New 
questions arise. Is the value gained worth any lofisec incurred? 
To what extent can other programs be expected to support a loss 
producing program? Which programs are to be combined to produce 
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a self supporting unit within the total framework of the college 
or university? Answers to these and similar questions form a 

very real part of any program evaluation and of necessity are 

unique to each institution. This paper will leave these answers 
to the more daring souls at each institution. The purpose here 
is to examine some of the methods and problems of conducting 
cost-income studies . 

Cost-income studies as developed by the Management Systems 
Office of Seniinole Connnunity College vary greatly in the amount 
of detail and depth but are precisely defined. The attention 
to definition allows the user the options of introducing new 
elements/ changing the elements, adjusting the mix of elements, 
or requesting greater detail. The studies are not purported to 
be precise measurements of anything. Rather, they are produced 
to illustrate, within the carefully defined constraints, the 
economic health of a program. 

Hopefully, the studies serve two functions. First, the 
studies can act as the indicator of the economic value of a 
program. This economic value used in conjunction with the 
other values needed in program evaluation allow for the most 
realistic appraisal of the present worth of the program. Second, 
if the studies show the program to be a marginal program or a 
loss producing program, the constraints used in the study and 
the elements of the study itself often indicate prescriptive 
measures for program health. Again, there must be a critical 
examination of the interaction of economic and educational values* 
Are the changes required to produce economic health too drastic 
to allow the program to operate in a meaningful manner? Can 
two programs be combined in such a way to improve the economic 
health of both without too great an educational loss in the 
trade off? What elements in the cost-income mix are amenable to 
adjustment? How much adjustment is required? What eJ^ments 
not in the study need to be investigated? In the proc=.>ss of 
this examination, programs can and have changed their character 
to such a degree that the financial picture is brought into a 
positive focus . 

Before actual cost-income studies begin, there should be 
an agreement with users as to the time frcime of the study, the 
definitions of the elements of the study, and the general method 
used in the study. Hopefully, this agreement will lessen, to 
some degree, the often violent reaction which follows the publi- 
cation of some studies. At the very least the agreement will 
provide a plausible beginning for the ensuing discussions. 
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The time frame is important because data need to be as 
current as possible. However^ the data need to be as accurate 
as-possible. At Seminole most cost-income studies are called 
for during Term II. This is the time of very difficult personnel 
and budget decisions. The studies most generally encompass 
Term I data only. Personnel^ payroll^ financial and student 
data are complete and checked to provide a sound base. The use 
of Term I data can create some problems. The extension of the 
information to a yearly basis is fraught with hazards and often 
requires a tentative follow-up of Term II data. Also, Term I 
is often a high enrollment, high productivity term and can paint 
a rosey picture which full year operations will not support. 
However, the use of prior year data or the estimation of full 
year operations pose too many uncertainties. The limitations 
imposed by using Term I data do not negate the usefulness of 
the studies if the users are well aware of the constraints and 
problems . 

Income elements include student income, state support, and 
any other definable income such as direct support from industry, 
government or other sources. Student income, unfortunately, is 
not clearly defined in a multipurpose, institution such as 
Seminole. Credit fees are twelve dollars per student credit 
hour. Non-credit courses vary from less than a dollar per 
student semester hour equivalent to well over twelve dollars 
per student semester hour equivalent. Term I 1975-76 data shows 
the Seminole Community College average was $8.03 per student 
semester hour. Agreement must be made on the student income 
base. Should all programs use the average income or the income 
unique to students within their area? The utilization of special 
fees such as lab or music must also be considered. 

State income introduces similar problems. The Florida 
Community College Division begins the proces=s of funding on 
differential values for each discipline. However, the inevi- 
table low level of funding and an estimated enrGl^nent base 
coupled with mid-year cuts due to insufficient state funds make 
it very difficult to establish a meaningful local deferential 
funding level. A simple solution is to divide the allocated 
state funds by the estimated total number of student semester 
hours to produce an average income er student semester hour. 
Again, any problems caused by this approach must be understood 
by the user. 

Other income presents no problems of detinit;vo:a< The 
only problem is the one of determining if such ^rrcome exists 
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and to what extent. if such income is provided on other than a 
regular basis per defined unit, this fact must become a part of 
the study, Seminole Community College has one program which was — 
a break even program in 1974-75 due to a heavy influx of industry 
funds. The program is now a loss program due largely to the 
removal of those industry funds. 

Expenses may be approached from two broad viewpoints. A 
budget or chart of accounts approach will use direct or instruc- 
tional cost, direct department overhead, cost of equipment and 
college-wide overhead. The cost accounting approach utilizes 
depreciation in place of equipment cost. 

In most cases the cost-income study is conducted within a 
very limited time frame. Also, the study should not be more 
costly than the possible program savings. For these reasons, an 
expensive and time consuming total audit is neither practical 
nor even beneficial. If current year data are used in the study, 
the problem of extending Term I data to the full year still 
remains. However, great care should be taken with the instruc- 
tional cost as this is not only the larger part of cost but 
also the most amenable to change. In rthe studies at Seminole 
Community College, the payroll records of instructors are matched 
with the individual teaching assignment to eliminate any salary 
which should properly be assigned to a different discipline or 
to non-instructional activities. Benefits to personnel are 
considered to be a part of the direct cost of instruction and 
the retirement and FICA payments made by the College on behalf 
of an instructor become a part of total instructional cost. 
Insurance and other benefits have been classified in the Florida 
Community College Cost Analysis and College-wide expenses and 
are treated as such in these studies. 

The College uses two separate approaches to determine over- 
head and presents both in the final report. The user has the 
options of using either approach, or since the elements of cost 
are clearly displayed, any mix of elements appropriate to needs 
may be chosen. 

A great debate sure to ensue with any discussion of cost 
centers around "marginal cost." The author has never been 
able to produce a working definition of marginal cost. Further, 
he has never been able to find the threshold at which full and 
marginal cost go their separate ways. He has, however, been 
informed by various program managers that almost the total 
offerings of some instructional components are able to operate 
on marginal cost. All other cost is being absorbed by some 
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seldom defined other component of instruction. 



Cost is displayed in such a way as to allow for recognition 
of the marginal cost concept. This cost display also provides 
data using current fund expenditures for Term I. Current fund 
expenditures are divided into supplies and services, and equip- 
ment expenses for the term. A minimal college-wide overhead is 
also charged. The data used in this configuration will produce a 
low figure by eliminating departmental administrative and support 
personnel cost. There is also a certain lack of stability of the 
data due to the uneven pattern of spending within the supplies 
and services and equipment budgets throughout the year. If this 
is known and understood by the user, adjustment can be made' if 
necessary. 

The second mode of displaying cost uses the same basic data 
on instructional cost combined with data from the latest cost 
analysis. For cost-income conducted during Term II of 1975-76 
departmental and college-wide cost data from the 1974-75 cost analy- 
sis are used. In the marginal cost method there are no charges 
for departmental administrative or support personnel* With the cost 
analysis based cost, -both of these costs are included. Unless 
there has been a drastic reduction in budget ^mounts for supplies 
and services, the past data seem to serve well and are more 
representative of full cost. The college-wide overhead for the 
past year will probably be low due to the effect of inflation 
and rapitJly raising energy cost. Seminole has been very near 
the bottom of the scale of community colleges in college-wide 
overhead per student semester hour and has not had the oppor- 
tunity to make any appreciable reduction in this spending. 

The use of cost data introduces another change that should 
be noted. Cost analysis uses a five year straight line depre- 
ciation rather than equipment cost. If the user requests a sub- 
stitution of equipment cost for depreciation cost, the cost 
analysis data are sufficient to allow the interaction. 

All cost data included in income and expense items use only 
the monies of the general current funds of the College. No 
federal or other non-recurring funds are used. Proposed legis- 
lation which would require the College to retain all depreciation 
of vocational equipment for replacement of that equipment only 
could require a new look at this component of the cost-income 
studies. If, indeed, the monies must be held in trust, new 
income and cost factors will need to be agreed upon. 

The study should not confine itself to the essential fiscal 
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data if other data are uncovered in the analysis. The required 
examination and checking of the enrollment base will often lead 
to an investigation of the patterns of course offerings or the 
composition of the student body of the program. Examination of 
currant expenditures may provide hints to in-depth examination 
of other accounts and in fact, other cost centers. A chat with 
the purchasing agent, the accountants, the college print shop 
personnel, and even the telephone operator are often useful. One . 
good source of information are maintenance and custodial personnel. 
There is often an unusual drain on these services by some programs 
that should be considered. There is no check list of personnel 
to see or sources of data. One must simply be alert to any clue 
and follow through. A suspicious nature may help but care should 
be taken not to waste time and effort for very little helpful 
information. Any extended examination should be undertaken 
only if there is a strong indication that a real cost or cause 
will follow. 

Enrollment patterns of an unusual nature should be noted. 
These patterns often indicate prescriptive measures for improved 
financial health. Such things as too many small sections within 
a set of multiple offerings of one course or an imbalance of 
full classes to small classes often appear. Examination of course 
titles can provide information about shifts in the thrust of the 
program. Programs may move from being largely a service to the 
entire college to a strong disciplinary approach. The discipli- 
nary approach may create a lov average class size and thus 
increase the cost while reducing income* In some programs course 
titles or descriptions indicate further examination of the types 
of students within the program. In one case, course descriptions 
led to the discovery that more than half of the student body of 
one program came from federally supported programs. This data 
had implications for staffing and long range commitment as the. 
funding was tentative. Any unusual student data would be 
beneficial to program evaluators. 

Whenever a program is only a part of a cost center, it 
would be worthwhile to examine the possibilities of the program 
using more than its proportionate share of the resources of that 
center. Are special support personnel assigned to the program? 
Would a check of some of the purchases for the cost center be 
appropriate? What does the purchasing agent say? Are there 
other programs or cost centers from which the program may draw? 
These are some of the concerns that need resolution. 

Unusual findings become a part of the "notes" of the reports. 
These findings should be presented as factually and simply as 
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possible so as not to imply any particular value and thus give 
bias to the main body of the report. As with the report itself, 
the "notes" should only serve as information to be iitibeded into 
other information for use in program retention, modification, 
or dissolution decisions. 

Appendix A gives an example of a cost study of a hypothetical 
program. As is often the case, the study provides as many ques- 
tions as answers. Using cost approach "A" the program will make 
its way. However, can it be justified as a marginal program of 
this degree? A look at equipment cost to date indicates a cure 
and a question. The cure would ho3.d these expenditures at 
their present level to overcome some of the loss displayed in 
option "B". The question is: How low can these expenditures 
be and what is now encumbered? Other information suggests 
possible further adjustment. Could fees be raised to improve 
income? An increase to $12.00 in fees would produce an additional 
$15,934.05 income. Will the present full-time staffing be a 
problem if the CETA program is withdrawn? If the student body 
of the program is changing to "support students", is the general 
character of the program also changing? Would a detailed study 
of the instructional cost which includes full-time, part-time, 
and overload information be helpful? As you can see, the 
information can be prescriptive, valuable, and madding. The pro- 
gram evaluators must now decicLe^.the needs for more information 
and the sources of that information. The evaluators must also 
decide the action or inaction which will follow. Program managers 
will often provide the necessary changes when they have the 
opportunity to examine and understand the implications of the 
report . 

The relatively large demand for reports and the questionable 
value of producing some requested reports has led to the develop- 
ment of a formula for quick, if not precise, data. The formula 
(Appendix B) has the advantage of producing indicators of 
problem areas without involved detailed examination. Basically, 
the formula uses the full cost approach of option "B" of the 
more detailed cost study. The choice and number of variables 
will allow for as much precision as does the full study. However, 
the formula has been found to have the greatest use as a broad 
indicator of program financial health in both a historical and 
predictive setting. 

The formula simply states, in algebraic terms, that the 
income must be greater than, or at least equal to, the expenses. 
The formula has the advantage of providing certain guidelines for 
what would be necessary to make a program self-sufficient. As 
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with any algebraic formula, the formula may be solved for any 
variable. However, general usage has shown that the student 
semester hour and the instructional cost have the greatest prac- 
tical usage. 

Using the data from the previous example (Appendix C, Part I) 
the formula yields S - 5,919 and P ^ 89,166. In other words, if 
cost were held constant, production would have to be increased 
by 545 student semester hours in order to break even. If pro- 
duction is held even, then instructional or other costs would 
have to be reduced by $9,211. 

The formula can be adapted to a variety of usages. Given 
the estisTiated salary expenditure for one section, the formula 
would produce a minimum economical class size. Assuming the 
income and expense data of the example and a salary of $1,200, 
the formula yields a minimum of 83 student semester hours 
(Appendix C, Part II) . If the cotirse were a 3 student semester 
hour course, then 28 students becomes the minimum class size. 

The formula usage is limited only by the availability of 
data and the imagination and needs of the user. 

The cost-income study and the program evaluation it supports, 
can be a time consiaming and painful operation. The financial 
exigencies and the program modifications or eliminations which 
will follow are not all bad. Education too often operates on 
momentum alone, carrying with it outdated and questionable pro- 
grams. There is a need to look at what we do and question it. 
The resources and energies spent in some programs could be much 
more productive if placed elsewhere. 

As my friend of the introduction might say: "Sometimes we 
got to do away with some of the things we got, so we can appreciate 
the things we gotta have". 
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Appendix A 



HYPOTHETICAL PROGRAM 
COST - INCOME STUDY 
TERM I 1975-76 



INCOME: 



SSH 
PTE 



5,365 X 9.03 (1) 
178.8 X 842.00 



48,445.95 
150,549.60 
198,995.55 



EXPENSES : A 

Instructional 98,376.75 

600 + 705 26,975.00 

710 5,125.00 

College-wide 53,650.00 



B 

Instructional 98,376.75 
Division Overhead 40,237.50 
College Overhead 69,637.70 



184,126.75 



208,251.95 



Based on proportional part 
of supplies and services 
(600+705) and equipment (710) 
Cost for 1975-76. $10 used 
as College-wide Overhead. 
No Departmental Administrative 
or Support Cost. 



B 



Based on 1974-75 cost analy- 
sis cost on non-federal 
funds. Program overhead 
$7.50/SSH. College-wide 
$12.98/SSH. 



NOTE: (1) Term I 1975-76 average income/SSH 

(2) 2,310 SSH created by 35 sections of CETA classes 

with average class size of 22. Average class size 
for remaining sections is 18. 
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COST - INCOME FORMULA 



Variable Listing 

S - Student semester hours (ssh) 

F = Student fee per ssh 

R = State funding per ssh 

D = Division Overhead cost per ssh 

C = College-wide overhead cost per ssh 

P = Instructional personnel cost in dollars (Include Retirement 
and PICA) 

Optional Variables 

W = Fee waiver cost in dollars (Use when waivers are responsibility 
of program) 

I = Special income in dollars (Lab or other special fees charged 
for program) 

E = Special expenses in dollars (Not covered in C) 

Formula with two alternate solutions « 

INCOME EXPENSES 
[(F • S) + (R . S). + ij - [(D • S) + (C • S) + P + E + w) - 0 

(optional^ 

F • S+R - S-D • S-C • S-P+ll-E-wl-O 

(F+R-D-OS-P + I- E^W^O 
Let X = F + R- D-.C ssh factor 
Solve for S 

^ P - I + E + W , . N. P 

S - X or wxthout options S - x 

Solve for P 

P-X-S+I-E-W or without options P ^ X • r 
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HYPOTHETICAL PRCX3R7VM 
FORMULA 
COST - INCOME 
STUDY 
PART I 

S = 5,365 
F = 9.03 

R = 842.00 -r 30 = 28.07 

D = 7-50 

C = 12.98 

I = 98,376.75 

X = 9.03 + 28.07 --7.50 - 12.98. = 16..62 
s ^ £ - 98,376.75 _ cqiq 

P ^ X • S = (16.62) 5,365 = 89,166 

CLASS SIZE STUDY 
PART II 

Salary 1,200 
Benefits 15% 
Total Salary 1,380 

S ^ ■ 83 ssh 

16,62 

83 f 3 = 28 students 
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ABSTRACT 



This paper analyzes the results of an instructional effective 
ness instrument completed by students on nine instructors in the 
Polk Community College mathe^atir.5 department. The findings show 
that the evaluation used diu not pinpoint an instructor's instruct 
ional deficiencies and/or virtues, one of the two purposes of the 
evaluation. To analyze the second purpose of the evaluationt a 
Duncan Range test was used and it was founa that the evaluation 
could not be used to determine jlear cut order of merit among 
ths instructors being evaluated. 
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INSTRUCTIONAL EVALUATION SYSTEM 
Int roduction and Background 

In the fall of 1971 the Vice President of Polk Community College 

Instituted a new irvstructlonal evaluation system. The purposes of this 
evaluation system were to help Instructors Improve their effectiveness 
and to establish an order of merit among instructors. The order of merit 
was a result of a Board of Trustees mandate for merit pay. 

The evalL'^tson system has two components, organizational effective- 
ness and instructional effectiveness. Each component has three parts, 
ratings or questionnaires. Organizational effectiveness ratings are 
completed by the instructor's dean, diyision chairman and department 
head. The three instructional effectiveness ratings are completed by 
the instructor's divis -^n chairman, his department head, and his students. 

Many questions have ^ven asked and many criticisms voiced about 
the system. How do the various sections of the evaluation correlate 
with each other? how do I know what my scores mean? how do I score 
in relation to others? do the system really do what it is suppose to 
do? are the students evaluating the course or the instructor? are a 
few of the questions asked. 
Purpose of Studj^ 

The purposes of the stuc|y are; 

1. To determine if the instructional effectiveness evalu- 
ation by students, pinpoints an instructor's instructional 
deficiencies and/or virtues. 

2. To determine if there is a clear cut order of merit in 
total scores on the instructional effectiveness question- 
naire completed by students. 
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The data for this study is from the instructional effectiveness 
evaluation completed by students for the instr.<ctors in the mathematics 
department in a fall term. 
Analysis for Purpose One 

The instructional effectiveness evaluation completed by students 
is divided into two parts. The first five questions are comp<)nents of 
teaching (as defined in this evaluation). A copy of the evaluation 
form is attached. To deteinine if the evaluation could pinpoint 
instructors' weaknesses, a one-way analysis of variance was run on the 
first five questions for each instructor (see table one). In eight of 
the nine instructors in the department there was not a significant 
difference in mean responses for the first five questions. For Instruc- 
tor number seven, an F of 8.44 was significant at the .01 level. 

(Table 1: Mean score by instructor for questions A, B, C, D, and E) 

INSTRUCTORS 





7 


2 


3 


4 


5 


6 


7 


8 


9 


A 


5.32 


5.13 


5.30 


5.53 


5.15 


5.09 


4.42 


4.14 


4.90 


B 


5.84 


5.10 


5.47 


5.42 


5.21 


5.33 


3.62 


3.97 


5.07 


C 


5.61 


5.26 


5.71 


5.51 


5.29 


5.64 


4.80 


4.50 


5.25 


D 


5.84 


5.13 


5.69 


5.67 


4.88 


5.43 


5.07 


4.32 


4.97 


E 


5.52 


5.01 


5.20 


5.44 


4.89 


5.04 


4.64 


3.86 


4.74 




0.71 


0.33 


1.74 


0.53 


1.25 


0.67 


8.44* 


0.77 


1.03 



F Statistic *Significant at .01 level 

Since there was not a significant difference in the response in 
eight of the nine instructors, the first five questions are not adequately 
pinpointing virtues or deficiencies of ..instructors. In fact, the five 
questions as stated might be condensed into one question. Also, the 
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question is raised; is there really no significant difference among 
these components of teaching or is the nature of the subject matter 
causing this result? 

When the F test was run for each instructor on the last ten 
questions, there was a significant difference at the .01 level on 
the scores for two-thirds of the instructors in the department (six 
instructors). The F's which were signlficmtly different are indi- 
cated in table two. The Duncan Range Test was then applied to the 
six cases. The questions were sorted by mean score in ascending 
order and Duncan's lines were drawn under the non-signifi_pant groups 
of questions. Table three gives these results. 

(Table 2: Mean score by instructor for questions F - 0) 



INSTRUCTORS 





1 


2 


3 


4 


5 


6 


7 


8 


9 


F 


4.07 


?.70 


3.67 


3.80 


3.46 


3.91 


3.00 


2.92 


3.18 


G 


4.18 


3.75 


4.10 


4.09 


3.90 


4.24 


3.49 


3.47 


3.71 


H 


4.03 


3.91 


4.24 


4.16 


3.89 


4.09 


2.88 


3.00 


3.43 


I 


4.23 


3.83 


4.40 


4.01 


4.23 


4.39 


3.91 


3.74 


4.28 


J 


4.19 


3.56 


4.39 


4.11 


3.82 


4.09 


3.33 


3.21 


■;.75 


K 


3.68 


3.68 


3.95 


3.92 


3.63 


3.54 


3.38 


2.89 


3 • 38 


L 


3.45 


3.68 


3.73 


3.94 


3.33 


3.40 


3.68 


2.83 


3.06 


M 


3.53 


3.60 


3.36 


3.60 


3.28 


3.56 


2.68 


2.65 


2.84 


N 


4.08 


3.77 


3.85 


3.92 


3.81 


3.82 


3.36 


3.50 


3.61 


0 


4.34 


3.96 


4.05 


4.04 


4.09 


4.05 


3.64 


3.61 


3.41 




1.99 


1.44 


6.35* 


3.02* 


6.86* 


1 .30* 


8.fi2 


3.11* 


9.80* 



F Statistic *significant at .01 level 
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(Table 3: Questions F - 0 in ascending order of mean score with 
Duncan's lines for non-significant ranges.) 

Instructor: 3 4 

Question: MFLNKOGHJI MFKNLIOGJH 

Duncan's lines: 



Instructor: 5 7 

Question: MLFKNJHGOI MHFJNKGOLI 

Duncan's lines: 



Instructor: 8 9 

Question: MLKFHJGNOI MLFKOHNGJI 

Duncan's lines: 



In table three it should be noted that questions M and F are 
in the losest non-significant range ror all of the six instructors 
and that question G, N, and 0 are also in a non-significant range 
for all of the instructors. This raises the question as to whether 
all six instructors have these same characteristics of if this 
result is characteristic of the subject matter. Since question I is 
the top question in five of six cases, can it adequately distinguish 
between instructors? 
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Analysis fo.- -'Vgl^^jSfe Two 

A total score on the instructional effectiveness questionnaire 
completed by students was determined by adding the mean scores for 
the first five questions and the adjusted mean scores for the last 
ten questions. The adjusted mean score was used since the range from 
high to low response is different for the two sets of questions. 
Table four gives the total score for each of the nine instructors in 
the mathematics departmer* The scores are arranged by instructor 
in ascending order. Although total scores differ significantly among 
instructors, a Duncan Range Test reveals that six instructors or two- 
thirds of the department, constitute a non-significant group. In 
table four, Duncan's lines are indicated below the non-significant groups. 

(Table 4: Instructors' scores in ascending order with Duncan's lines 
for non-significant ranges.) 

Instructor: 

879256431 
Score: 

52.20 wj.OO 59.90 62.50 62.90 66.20 67.10 67.70 68.50 
Duncan's lines: 



The instructors might be ranked in order of score, but this 
evaluation instrument can not be used to establish a clear cut order 
of merit since there was not a significant difference between the six 
top scores of nine instructors' scores. Also note that there was 
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not a significant difference between instructor nine and Instructors 
two and five of the top six instructors. 

If merit pay is ta be given on the basis of this instrument (or 
if this instrument is a part of merit pay consideration), then who 
should receive it? If it is the top six instructors then what about 
instructor nine? He is not significantly different from instructors 
two and five. Also, as a result of these non-significant differences 
the question is raised, do these instructors possess similar traits 
or are students' responses due more to the nature of the subject matter 
rather than to the peculiar traits of an instructor? 

When the individual scores for instructors by course were examined, 
(see table five) it was interesting to note that instructor nine has 
the next to highest score in MTS 101 but the lowest score in MTS 111, 
while instructor five had the lowest score in MTS 101 and highest score 
in MTS 111, but that both scored high and approximately the same in 
calculus (MTS 252, 253). When the F statistic was computed for 
instructors 5 and 9, it was found to be significant at the .01 level. 
In fact, in five of the. seven instructors teaching more than one course 
there was a significant differejc.e..(.0&), between the courses taught. 
This indicates that the course assignment makes a difference in an 

instructors evaluation, i.e., the order of merit can be effected by 

\ 

the peculiarities of the course. It is also noted that three of the 
four scores in the seventy's ockr in higher level courses, fortran, 
225, calculus I, 252, and calculus II, 253. 
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(Table 5: Instructos' scores by course) 



COURSE 

100 101 102 105 111 112 113 145 225 252 253 265 

1 68.5 

2 62.5 62.2 62.5 

« 

3 (8.5 61.0 75.2 

' . 73.4 66.4 

S I 5 57.9 65.8 70.2 
c 6 66.2 

M 

7 57.0 60.0 46.9 

8 62.6 48,1 

9 64.4 52.3' 70.9 



Conclusions and Recommendations 

This study indicates that the instructional effectiveness form 
completed by students does not pinpoint an instructor's deficiencies 
and/or virutes, and that there is not a clear cut order of merit in 
the scores of instructors. In fact, there are indications that 
students may be evaluating : course as much as they are evaluating 
the instructor. 
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USE NO. 2 PENCIL ONLY pOLK COMMUNITY COLLEGE 

Make your mark as long as the pair of l.nes. and complately ivsTRUCTION VL EFFECTIVFVESS 

fill the area between the pair of lines. If you change your l.>-^iKULi IO^AL LFFECTIVENESS 
mind, erase your first mark COMPLETELY. 



(FOR STUDENTS) 



SECTION NUMBER 
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:iZ:: 
















zOzz 




:2:: 




ZA--Z 










=9:: 








::«:: 




:0:= 




=2zz 






















r:*:: 





DEPT. 



FORM 12 



zdSc: rzBrr trJrr rrft: rA: 
rA= rzfc 



zAz z:^ zzltz zz^z zz%z 
INSTRUCTOt 

:z5:: rrS!:: rrft= rrfcr 



A. Has command of the subject, presents material in an analytic way, contrasts 



points of view, discusses current developments, and relates topics to other 
areas of knowledge. 



LOW 
SCORC 

zztzz rrfc: 



PCC HIOH 

NORM tcoac 

=:i: =r*: zz&z z:ft: rrfc 



B. Makes himself clear, states objectives. summarizes major points, presents 
material in an organized manner, and provides emphasis. 



C. Is sensitive to the response of the class, encourages student participation, 
and welcomes questions and discussion. 



D. Is available to and friendly toward students, is interested in students as 
individuals, is himself respected as a person and is valued for advice not 
directly related to the course. 



ziizz zzZz 



zD^z zz^z zz&z zzj^z 



zzZz 



SI*: zztz zzjzz 



zz^z -3:: zz^z zzjzz 



E. Enjoys teaching, is enthusiastic about his subject, makes the course exciting 
and has self-confidence. 



zztzz :=fc: 



-S: ::*= zz^z zzfzz 



F. Has increased my appreciation for the subject. 



LOW 
SCOHC 

zztzz 



MC HIQH rAly**^ 

NORM 8C0RC oottxuLX 

-S: : 



G. Keeps well informed about the progress of the class. 


zztzz zzZ: 


zz^z 






00«i»T «•«• 


H. Anticipates problems and makes difficult topics easy to understand. 


"2:: 










1. Speaks clearly. 


::2:: 


"if: 






00(1 MT 
BOUT KM* 


J. Quickly grasps what a student is asking cm* telling him. 


=-r-- zzzz 








et>M*T rao« 


K. Presents the^aesiS^fep and emotional values of the subject 


zzr-z zzZzz 




zz^z 




MIIMT 

OMrr tiio* 


L. Relates class topics to students' lives and experience. 


"4" 


:=4:= 


zzc:z 




OOUT SKO* MM 


M. Gives interesting and stimulating assignments. 


r=t= 




zz^z 




•«SS MOT 
BOH'T titO* MM 


N. Gives examinations that require creative, original thinking. 


zzZz 


=15:: r=4=: 


zz^z 




«pLT Oil 

don't SM* 


0. Gives examinations that have instructional value. 


zztzz ZZZZ 


r^-. 


zz^z 




»<XSMt «^ 
eON'T KHO« 


ITEMS BELOW MAY BE USED UNDER CERTAIN CIRCUMSTANCES. DO NOT MARK UNLESS INSTRUCTED TO DO SO ~ 
IF YOU HAVE ANY GENERAL COMMENTS TO MAKE ON INSTRUCTOR, WRITE THEM ON REVERSE S^ ~ 


P. "F" =--2== :=3:: '=4=- -5^: r-T:: 


V. 1:3:: 


==5: 


-S- 


zz^z 




Q. =--2== z^-. zzifz .zSiz zz^z zzpz 


W. 
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Faculty Salcury Increases 
and Evaluation of Selected Performance Vcuriables 



T» Wayne Keene 
University of South Florida 

ABSTRACT 

The study was based on a university of ahont 800 faculty offering 
baccalaureate, Masters, and doctorate proijrams. The purpose was to deter- 
mine the relationships between recommended salaury increases and evaluation 
of pexfonnance-. Salary increase proposals were submittcKi for faculty by 
department chairpersons. Among other items of information, the proposals 
cont4iIned three categories of data: (1) recommended salary increases for 
t-isaching, research, and service, (2) workload assignment distributed by 
percentages, and (3) chairperson's scale evaluation of performance. 

Correlation analysis was applied to the Vciriables for faculty in the 
discipline categories of Business^ Education, Engineering, Fine A. "e. 
Humanities, Natural Sciences, and Social Sciences. Correlations between 
merit salary increases for teaching and evaluation of teaching and evaluation 
of overall quality were about the same moderate Cpositive) . Correlations 
between saleury increases for activities other than teaching and evaluations 
of those activities differed somewhat from teaching « Merit increase for 
research correlated (positively) with evaluation of research to a considerably 
higher degree than was the case for teaching. In general there was higher 
correlation between the different variables with respect to evaluation than 
between salsury increases and evaluations of the merit increase categories. 
Conclusion: The Correlations were weak, enough to cast doubt on the value 
of chairpersons' evaluations in granting salary increases. 
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FACULTY SALARY INCREASES AND EVALUATION 
SELECTED PERFORM?sKCE VARIABLES 

Introduction 

There has been increased debate in recent years about the pay of 
college faculty members. No doubt the rapid growth of collective 
bargaining in education has served to agitate the issue. 

Teaching, of course, is the primary task of college faculty. However, 
there are two other important areas of activity in which faculty engage: 
research/creative activity and service (public and institutional) . These 
three major traditional areas of college faculty activity aza operat /e to 
some degree in all higher education, though with varying degrees of emphasis 
(and reward) among universities, four-year college and community colleges. 

An exploratory study of the faculty salary increases proposed by college 
chairpersons for the year 1972-73 wa conducted at the University of South 
Florida. The year 1972-73 was celerteci because that was the first year in 
which salary increase proposalc were based on formal uniform evaluation 
procedures. Numerous descriptive statistics and several measures of 
relationship were applied to the data. A major matter of interest was the 
extent to which recaranended salary increases were reflections of performance 
evaluations made by chairpersons. Were large narit increases for teach ir^'? 
matched by high evaluations of teaching effectiveness? Did a given 
evaluation score for research earn more increase than the same score for 
teaching? Were high evaluations for service recognized by cor:, esponding 
increases? Di 1 increased assignment in an area tend to inflate the evaluation 
score of that area? The purpose of the study was to answer these and siitiil^.r 
types of questions, search for improved approaches to evaluation and reward, 
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and establish a base for further research* 
Procedures 

This discussion will emphasize correlatic nal analyses involving evaluation 
ratings and salary increases, especially in the teaching activity area. Wbrk- 
load assignment will be considered only as necessary for perspective.^ 

A salary increase proposal form was submitted for faculty members by 
chairpersons to deans thence to the office of the Vice President for Academic 
Affairs, The forms inclxided data ^In addition to numerous other items) as 
follows: 

1. Recommended merit increase amounts for (a) teaching, (b) 
research and creative activity, and (c) service (professional 
university, public). 

2. Chairperson's evaluation (5 point scale: 1-lowest to 5-hi_,aest) 
of (a) teaching effectiveness, (b) research and creative 
activity, (c) service, (d) advising, and (e) overall qu lity. 

3. Workload assignment in percentages-of-total-workload for the 
previous year for (a) teaching (graduate; undergraduate) , (b) 
research and creative activity, (c) service, and (d) advising. 

Twelve -month faculty, part-time faculty, faculty who had submitted 
resignations, and ranks other than professor, associate professor, 
assistant professor, instructor, and lecturer were excl\aded. The 
remainder, 635 9-mon . faculty, were the subjects of the study* 

Findings 

Figure 1 displays the Pearson r correlations of variables discussed 
i;i this paper. 
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Figure 1 

CORRELATION BETWEEN TEACHING INCREASE AND SELECTED VARIABLES 
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Correlation between saleiry increase for acMng and evaluation of 
teaching perfonricince is present but only y nte, + .40. It is interesting 
to note that tlie correlation between teaching increase auKi overall evaluation 
io higher, though only slightly so, + .42, There is a noticeable drop from 
th^.se levels between teaching increase and other variables — to + .27 and 
+ .24 for service and research, evaluations to virtually no association with 
the assignment vcuriables. The higher correlation between teaching increase 
and teaching evaluation is expected, or certainly hoped for. The similar 
higher level of correlation with overall quality suggests that the raters 
tended to associate overall quality with teaching performance. 

It is interesting to compeire research and service relationships in 
light of the teaching data in Figxire 1. Thr correlation between incrv?ase 
for research and evaluation of research r formance was moderate, + .49. 
The correlatiom between reseeurch increase and other variables were low, 
with one exception. Correlations between research increase and teaching 
'•ind service evaluations were both - .03, indicating virtiially no association. 
Overall quality ev-»luation was a different matter. VThile considerably less 
than the teaching increase - quality orrelation, it was present: to a slight 
degree — + .23. Noteworthy was the correlation between research increase 
and research assignment, v .53, which is higher than the correlation between 
reseeurch increase and research evalue.tion. This suggfec^r.s the possibility 
that in research the chairpersons tended to recommend increases based on 
assignment rather than performance evaluation. 
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Increase for service and evaluation of service were moderately corre^ 
la ted, + ,40. Correlations between service increase and other evaluation 
variables were + .15 for evaluation of research and + 730 for evaluation of 
overall quality. This places service about halfway between teaching and 
research C+ .42 and + .23 respectively! so far as correlation coefficients 
between those increases and overall quality evaluations are concerned, it 
is apparent that overall qusility was more closely associated with teaching 
than with research and service so far as salary increases are concerned. 
The correlations between service increase and the assignment variables were 
- .12 for teaching assignment arid + .34 for service assignment. The fairly 
close correlations between service increase and service evaluation and 
assignment (+ .40 and + .34) suggest that the chairpersons were confounding 
evaluation and assignment ratings when reconnner iing salary incrciases for 
service. 

It should be noted that evaluations were : '^aled jra.inal data (1, 2, 4, 
5, NA-not applicable) while increase and asrigyirc: t ^v-^rer fcvr all practical 
purposes, continuous data wide range. Th'.-. ./::umstance is iaown to 
affect some statistics?, r--^^l" to niainterpretation. However, Sta^xiran's 
rank-order correlation lu^ -o.r.ntialJ y tc the same results as the Pearson r,' 

Additional Considerations 

Table 1 contains frequency dat<?. of association with respect to salary 
increase and evaluation rating foi .f^^^ching. 
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Table 1 

FREQUENCY OF SALARY INCREASE BY INCREMENT INTERVAL 
AND PERFORMANCE EVALUATION: TEACHING 





0 - 


200- 


AOO- 


600- 
7qq 


800- 1,000- 

qqq i.iqq 


1 ,200- 
1.399 


1,A00- 

1.599 


1 ,600- 
1.793 


N 




5 


16 


57 


57 




A 


3 




1 




168 


28.6 




27 


109 


79 


12 


1 


2 








230 


39.1 


3 


A2 


79 


28 


3 












152 


25.9 


2 


1? 


1? 


2 














3A 


5.8 


1 


3 


I 
















A 


0.7 


N 




255 


166 




5 


5 


0 


1 


0 


588 




% 


18. 2 i 




28.2 


7.7 


0.9 


0.9 


0 


0.2 


0 




100.0 



185 



'69 



ERIC 



The evaluation score, "not applicable" is excluded from the data and 
computations. The data in Table 1 permits additional analyses of associa- 
tion. It provides added perspective to the Pearson r correlations mentioned 
previously. The difference between the 635 faculty in the sttidy and the N 
values in Table 1 is due to the requirement of matched pairs/ The latter 
excludes those who received a "not applicable" evaluation score, even if an 
increase was proposed. 

There were 65 faculty received no increase for teaching activity. 
Of these, 24, or 37%, were evaluated as above average (evaluation scores 4 
and 5). By way of compar ' son with the data of Table 1, 17 1 ox those who re- 
ceived no increase for research were evaluated as above average in research. 
The corresponding figure for service Is 31%. 

The above average evaluation matched by asero ir. rease cases are not easy 
to explain. This would seem particularly true in view of the fact that the 
highest percentage bt' no-increase cases matched by above-' verage evaluation 
occurred in teaching rat' ar than research and service. Could it be that 
high evaluations are used as a substitute for salary increases? If so, 
why did it occur more frequently in teaching than the other two major 
categories? Was it because chairpersons were inexperienced in relating 
formal evaluation data to salary increases? The answers to these questions 
are not suggested by the data of the study. 

The pattern of "piling up" of frequencies in Table 1 helps explain +:he 
general lack of high correlations between activity category and evaluation. 
The -3^,s?ociations appear to tend toward curvilinear relationship rather than 
the linear relation measured by Pearson r. The small number of evaluation 
values compared to the range of salary increase introduces problems with 
respect to parametric statistic; .1 procedures. Distribution-free statistics 
vrould seem to be a promising area of exploration for association patterns 
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experiencei with the variables such- as those in this study. 

Siainmary and Recommendations 

An ^exploratory study of salary increase proposals for 635 full time 
ninenronth faculty was conducted at the University of South Florida. The 
study was based on data contained in salary increase proposals irfxich 
included (1) recommended merit increase amounts for teaching r resecirchr and 
service; (2) chairperson's evaluation of teaching effectiveness, reseeurch, 
service, and overall quality; C3) percentage workload assignment in teaching, 
research, and service. 

Correlational analysis was applied to the two ^actors evaluation and 
salary increase in three categories of activity — teaching, research, and 
service. The correlations sr^rnmcirized here are Pearson r. Positive moderate 
correlation (+ .40) was found between teaching increase and teaching 
evaluation and between teaching increase and overall quality evaluation 
(+ .42) . Slightly higher positive correlation was found between research 
increase and research evaluation C+ -49) . There was higher correlation 
between research increase and research assignment C+ •SB) than between 
inciease and evaluation {+ .49) . The correlation between service increase 
and service evaluation was moderate C+ .40) , and between service increase 
cuid overall quality evaluation '.he figure was + .30. In general, correlations 
were present and positive, but only to moderate degree. 

A two-way contingency table of evaluation scores and S2d.ary increases 
intervals suggested that the relationship between these two variabl.s was 
ctirvilinear rather than rectilinear. This indicates the need to explore 
other measures of relationship between evaluation and salary increases if 
such measures are to have value for equitable decisionnmciking . 
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It is recommended that further research. Cuicluding non-peirametric and 
curvilinear techniques] be conducted in the area of faculty performance 
evaluation methods as related to faculty accountability and revraurd systems. 
Particular attention should be given to evaluation methods which involve 
scaling and weights. Provision should be made for evaluation scales or 
devices on which equivalencies can be established airvong various programs 
or discipline areas. In addition a second dimension-weighting-should be 
provided for the * *c categories and sub-categories of professional 

activity, such a , research^ service, advising, professional 

development and renewal, and the like. This is needed so that faculty from 
different instructional fields or organizational units can be compared on 
a standardized basis while maintaining different emphasis among the several 
categories. In-service training should be provided for chairpersons and 
deans where saliLLV recommendations are to be related to foirmalized ratings 
of performance. 

This discussion has concentrated on monetary rewcurd in the foirm of 
salary increases. Other types of reward, however, are involved. These 
include promotion, retention, tenure, type of assignment, and the like. 
Opportunities for ocjaitable professional cidvancement and c alopment and 
their corollary - productive activity - depend on reliable and valid 
systems of evaluation and reward. V/e must develop and continuously evaluate 
such systems. 
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Footnotes 



Extensive exploratory statistical emalyses of other data from 
the increase proposal forms, including workload assignment variables, 
were conducted. These include measures of central tendency and vari- 
ability, analysis of varicmce, association measures such as Chi Square, 
Phi, Contingency Coefficient, Kendall's tau, Gamna, Cramer's V, Somer;s 
D, Spearman Rho, and frequency distributions. Those interested in 
further details should contact the author of this paper. 



Spearman Rho coefficients were as follows: teaching increase/ 
evaluation + .39, increase/overall quality + .41; research — increase/ 
evaluation + .60, increase/overall quality + .29; service — increase/ 
evaluation + ,46, increase/overall quality + .32. The Speeunnan corre- 
lation efficiency is about 91% when compared to Pearson r.. 
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Abstract 



A case Is made for the Involvement of the Jn^tKut lonal researcher 
in instructional, faculty, and/or organization dev^oloprr^jint programs In 
higher education* particularly in the areas of needs assessment and 
program evaluation. The authors describe a research study that was 
designed to assess the commitment of faculty and department chairmen 
at the University of Florida to the utilization educational tech- 
nology. The study is an illustration as to how institutional reseaf&h 
can play a vital role in instructional development programs. 
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The Role of Institutional Research in Instructional Development: 
The Emergence of a New Model at the University of Florida 

In the last 5 years we have seen the emergence of a new Interest 
in staff or professional development in Arjierica's higher education 
community. Numerous models for staff development (SD) in universities 

and community colleges are being proposed (Gaff, 1975; Bergqulst and_ 

Phil Hps, 1975; Mammons, 1975; Richardson, 1975). Gaff ?n his book 
Toward Faculty Renewal identified three dsffe;6nt staff development 
approaches: f«icu!ty devel op-Tit'-; t ^ hisfcruct ional development, and 
organizational development (1975, p. 9). The position taken in this 
paper is that there is a role for institutional researchers In this 
new movement, particularly in th^. area of the identif ication of staff 
development needs in a college or university. A research study con- 
ducted by the Office of Instructional Resources at the University of 
Florida in the Fall of 1975 will be used to support this position. 

Two major areas of greatest need in present staff development 
programs throughout our country are for: (1) effective needs assess- 
ment procedures and (2) effective evaluation programs. Institutional 
research offices and officers could provide valuable assistance In 
both of these areas. In the area of e^valuation, the institutional 
research office could assist the Director of Staff Development in 
collecting data from students to determine whether or not particular 
instructional improvement projects make any difference in terms of 
student learning. This is but one example of a type of evaluation 
assistance that could be offered by Offices of Institutioiial Research. 
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But let us turn to the area of needs assessment. Gaff found In 

his questionnaire survey of 54 directors of college and university 

teaching Improvement or staff development centers that 

Twenty-seven directors said they have 'conducted a systematic 
survey or other analysis of faculty Interests and need In 
regard to Improving their teaching'; on the other hand, 
twenty-eight said they had not done that. An examination 
of the actual procedures used revealed that with few exceptions, 
the kind of analysis most often used has been a short general 
questionnaire to determine faculty Interest In possible center 
programs. Although this procedure may be quite useful. It 
does not necessarily reveal what faculty need to learn 
(Gaff, 1975, p. 127). 

It would appear from this research that many faculty development centers 

are not conducting careful needs a:>sessments In staff development. 

Let us now describe whsj the Office of Instructional Resources 

has done this past year In the way of a formal Instructional -'needs 

analysis" at the University of Florida. We think our study illustrates 

the Importance of Institutional research in the whole Instructional 

development process and provides a possible model for staff needs 

assessment that other Institutions of higher learning may want to adopt. 

Nature Of The Study 
The purpose of this study was to assess the commitment of faculty 
and departmental chairmen of the University of Florida to the utili- 
zation of educational technology.' Commitment was measured In terms 
of attitudes toward educational technology and knowledge of Instruc- 
tional programs. Specific questions which were asked in the research 

^Thls study was made possible by a grant to the University of Florida 
by the Alfred Sloan Foundation. 
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were : 

(1) What are the attitudes toward educational technology of 
the faculty and department chairmen? 

(2) What are the instructional problems and needs of the 
university identified by faculty and department chalmien? 

(3) What are the existing instructional resources of the 
university identified and utilized by faculty and 
departmental chai rmen? 

(4) How can a measure of openness or willingness to change 
on the part of faculty and department chairmen be 
determined? 

Procedures 

An interview schedule and an attitude scale were designed and 
administered to selected chairmen. Also an open-ended questionnaire 
and the same attitude scale were administered to faculty. The data 
gathered from these instruments were then coded and analyzed. 
Population 

A listing of 123 departments, names of department chairmen, 
number of faculty within the departments and number of student credit 
hours generated by that department was obtained from the Office of 
Academic Affairs. Non-teaching departments end departments with less 
than seven faculty members were deleted. A total of 80 departments 
became the population from which data were obtained. 
Instrumentat ion 

Three instruments were developed for the study and were designed 
to meet the following objectives: 

(1) To identify the instructional problems and specific needs 
of departments as perceived by faculty and department 
cha i rmen. 
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(2) To identify present, planned, and desired solutions to 
problems perceived by department chairmen. 

(3) To catalog the human and non-human resources available 
to departments as perceived by faculty and department 
cha I rmen. 

{k) To assess attitudes toward educatiooal technology as a 
measure of openness or willingness to change.' 

Interview Schedule for Department Chairmen . An Interview schedule 
was developed by writing Items relating to objectives. Drafting the 
schedule required four revisions before completing the final Instrument. 
Each of the major drafts was field tested by each of the study's inter- 
viewers. 

Faculty Questionnaire. The faculty questionnaire was a one page 
subset of the items found on the Interview schedule. Faculty were 
asked to list Important human and non-human resources, to nominate 
colleagues who they considered to be Innovative In instruction, and 
to list their Instructional problems ^nd needs. The only demographic 
information requested was rank and years. in the department. 

Attitude Scale . Twenty-four Items were developed from the objec- 
tives and goc^ls of the project to assess a faculty or chairman's 
attitudes toward the utilization of educational technology. Educational 
technology was defined to include not only the use of instructional 
materials and/or electronic equipment, but also the systematic design 
of Instruction. The rationale for each item wcs 'rhat it was to elicit 
an attitude or feeling, and not be a statement of fact. 

'copies of these three Instruments are available from the Office of 
Instructional Resources, University of Florida, 450 Library East, 
Gainesville, Fla. 32611. 
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A Likert format was followed where each item was a clear, concise 
statement, and the 2k items, 5 negative and 19 positive were randomly 
distributed throughout the scale. The responses were scores 1 to 5. 
A very favorable attitude toward the utilization of educational tech- 
nology was assigned a vaFue of 1 (strongly agree), and conversely a 
very unfavorable attitude toward the utilization of educational tech- 
nology was assigned a value of 5 (strongly disagree). A value of 3 
was assigned to a neutral or no opinion response. Highest possible 
score was 100, the lowest, 20. 

To assess the reUabiUty of the 2^- Item scale, a sample of 
faculty chairmen, who were a part of the target population, as well 
as several advanced graduate students who were fn measurement and 
statistics classes responded. The total number sampled was 39. The 
results of Hoyt's reliability coefficient showed the 2if-;tem scale to 
have a reliability of .73. 

An item analysis was made using item correlations with a person's 
total score to obtain the relation of an item to the entire scale. 
From the correlation matrix four Items were in question. These items 
exceeded a statistically significant level of .05 e.g., .10 to .30. 
It was decided to remove these four Items to be administered to faculty 
and chairmen at the Unfversity of Florida. The 20-item scale contained 
7 negative, 13 positive Items. 

Dcta Collection 

Each department chairman first received a letter which briefly 
explained the nature of the project and then was contacted by telephone 
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to obtain a personal interview. Only one chairman directly refused 
to be interviewed; three others never made themselves available. 
During the interview, an interview schedule was followed and the 
chairman was asked to respond to ti^e self-administered attitude 
instrument. The percentage of response was 95^- 

Faculty were asked to respond to the four- item, open-ended 
questionnaire and the attitude scale. Several procedures were used 
to solicit this information. Some chairmen who held frequent regular- 
ly scheduled departmental faculty meet ings permitted us to attend 
these meetings and faculty responded to the instruments at that time. 
A few chairmen were willing to call a meeting specifically for this 
purpose. Response was high for Loth these groups. Other departments 
held meetings much more infrequently; in some instances only once a 
quarter. In these departments we attempted to gather data by placing 
questionnaires in faculty mail boxes or sending them through campus 
mail. This procedure was extremely ineffective and often produced only 
a one or two percent return. A total of k55 faculty responded to the 
questionnaire and attitude measure. 

In summary, departmental chairman data which was collected through 
personnel interview is fairly complete. Faculty data ranges from par- 
tially complete to non-existent. 

Data Analysis 
Interview Schedule and Faculty Questionnaire 

Responses to the interview schedule and questionnaire were cate- 
gorized and a codebook for numerical analysis was developed. Nearly 
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^00 separate categories were Identified. A hierarchy of response 
categories was developed so that a respoj-^se could be easily coded. 
These items were for the most part mutually exclusive, and little 
overlap occurred. 

Each interview schedule and questionnaire was coded on optical- 
scan sheets by the interviewers. A canned computer program statis- 
tical package for the Social Sciences (SPSS) was utilized to compile 
the data by categories and departments. A portion, of SPSS, "Crosstabs", 
was used to examine relationships among chairman responses to pre- 
established categories. 
Attitude Scale 

The 20-item, revised attitude scale was completed by 76 department 
chairmen, and k55 faculty members. Their results were analyzed separately 
A program using Guttman's Reciprocal Averaging was used to obtain a score 
for each respondent. The scoring was reversed for negatively-phrased 
items. Responses were removed from the analysis If more than 5 of the 
20 Items were omitted. Fifteen responses had to be discarded, two of 
which were chairmen. The scores were used to select two groups of 
faculty and chairmen: those who had very favorable attitudes toward 
the utilization of educational technology, (e.g.), scores below 1 
standard deviation from the mean; and those who had very unfavorable 
attitudes towward the utilization of educational technology, (e.g. ) , 
scores above 1 standard deviation from the mean. By selecting + 1 
standard deviation from the mean approximately 32% of the sample or 
about 16% of the sample at each end of the continuum (favorable vs. 
unfavorable attitudes) were identified. 
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Resul ts 

The results of the data analysis are presented in two sections. 
The first presents a university-wide profile of chairmen's responses 
to the interview schedule. The second section reports the analysis 
of the data as a classification of departmental chairmen and faculty 
as being open or closed to Instructional Innovation and change, 
University-Wide Profile 

These data represent a description of the department chairman 
responses to the study's interview schedule. It provides a unlversity- 
wide profile of the following factors as perceived by chairmen: 

(1) human and non-human resources available to departments; 

(2) Instructional problems and needs; 

(3) types of faculty development or improvement activities 
used by departments; and 

{k) emphasis given teaching, research and service in tenure 
decis Ions. 
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Table 1 

Summary of Chairmen Responses 
Identification of Most Important Non-Human Instruct ionaS Resources 



Category Chairmen (N«76) Percent of 

Responses Total Responses 



1 . 


Library 


26 




7.2 


2. 


Classroom Hardware^ 


128 




ISA 


3. 


Classroom Software^ 


131 




36.2 


i». 


Research and Field Equipment 


35 




9.6 


5. 


Instructional Laboratories and 










Faci I i t ies 


h2 




11.6 






362 




100.0 



^Classroom softv/are is defined as instructional materials used in cla 
room settings; classroom hardware is defined as the equipment used in 
these settings. 



Table 1 summarizes the chairmen's responses to the question: "What 
are the most important non-human instructional resources available to 
you and your staff for the improvement of instruction in this depart- 
ment? From Table 1, one can see that the 76 chairmen mentioned class- 
room software and hardware most frequently as being their most important 
non-human instructional resources. This finding suggests a university- 
wide interest in the continued and expanded use of educational technology. 
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Table 2 



Summary of Chairmen Responses 
Identification of Most Important Human Resources 



Category Chairmen (N-76) Percent of 

Responses Total Responses 



1 . 


Faculty and Students 


108 


35.9 


2. 


Staff 


hS 


16.3 


3. 


Resource Centers 


73 


2A.3 


h. 


Conni ttees 


7 


2.3 


5. 


ProfessJonaJ Assistance 




13.6 


6. 


Instructional Strategies^ 


23 


7.6 






301 


100.0 



Instructional strategies are defined as computer- Involved Instruction, 
gaming and simulation techniques, demonstration, etc. 

Table 2 summarizes the chairmen's responses to the question: 
'•What are the most ImpoVtant human resources available to you and 
your staff for the Improvement of Instruction In this department?'* As 
one might expect chairmen felt that faculty and students were their 
most important human resources (35.9% of the responses). It is Inter- 
esting to note that the second most important resources mentioned was 
resource centers (2^.3%)- .Apparently chairmen perceive resource centers 
such as the Office of Instructional Resources as being an important 
resource for their Instructional program. Only a small percentage of 
the responses were in the category of instructional strategies (7.6%). 
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Table 3 on page 12 provides a profile of the major insuructional 
problems facing the University of Florida as perceived by university 
department chairmen. The major Instructional problems appear to be: 
(1) too many students (27.7*), (2) workloads too large (27.7*), (3) 
lack of funds for faculty (26.3*) and {k) lack of classroom space 
(21.1*). Major problems facing the University of Florida do not 
center around a need for more equipment and materials. Instead, the 
greatest problem appears to be an overworked faculty. This would 
suggest that future instructional projects at the University should 
provide faculty with reduced workloads and released time so that they 
can plan, implement and evaluate their instruction improvement projects. 

Table k on page 13 provides a profile of some of the major chair- 
men responses to the question: ''At this time, what are the major in- 
structional needs of your department?" The major instructioncil needs 
at the University of Florida appear to be: (l) more equipment and 
materials (^6.1*), (2) more funds for faculty (3^.2*), (3) more funds 
for facilities and hardware (25.0*) and {k) more classrooms properly 
equipped {Z2.h%) . The high response for more equipment and materials 
does not fit with the problems mentioned in Table 3. However, It does 
fit with the importance chairmen had previously attached to classroom 
hardware and software as an impoi^tant instructional non-human resource. 
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Table ^ 

Summary of Responses 
Frequency Analysis of Instructfons? Problems 





Category 


Number of 
Responses 


Percent of 
Chairmen 



1 . Space 

Lack of classroom space 

Inappropriate laboratories 

Lack of properly equipped classrooms 

2. Equipment and Materials 

I nadequate 
Not enough 

3. Lack of Funds 

For facilities and hardware 
For faculty 

For graduate assistants 
For support personnel 

h. University-Wide 

Political concerns 

5. Library 

Lack of appropriate materials 

'6. Faculty 

Lack of faculty cooperation 
Workloads too heavy 

7. Students 

Too many students 



16 
9 

15 



8 
12 



U 
20 
8 
iO 



8 
18 



18 



21.1 
11.8 
19.8 



10.5 
15.8 



18.4 
26.3 
10.5 
13.2 



6.6 



6.6 



10.5 
27.7 



27.7 
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Table k 



Summary of Chairmen Responses 
Frequency Analysis of Instructional Needs 







Category 


Number of 


Percent of 






Responses 


Chairmen 


1. 


Space 








More classroom space 




18.4 




Appropriate laboratories 


10 


13.2 




Classrooms properly equipped 


17 


22.4 




AODrODriate ^pHpHuI Inn nf r^laccronm 


space o 


in c 


2. 


Equipment and Materials 








Adequate materials 


11 


14.5 




More equipment and materials 


35 


46.1 




Better service from media centers 


10 


13.2 


3. 


Funds 








For faci 1 it ies and hardware 


19 


25.0 




For facul ty 


26 


34.2 




For graduate teaching assistants 


8 


10.5 




For support personnel 


14 


18.4 


k. 


Facul ty 








More faculty cooperation 


6 


7.9 




Reduced workloads 


6 


7.9 




Instruct ion 








Develop self --paced instructional mater ials9 


11.8 
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Table 5 



Summary of Responses Describing Departmental 
Instructional Improvement Activities 



Improvement 
Activity Category 



Number of 
Responses 



Percent of 
Chairmen 



Conferences/Workshops 

Informal Assessment 

Col loquIa/Semlnars/Guest Lecturers 

Fjculty Meetings 

Access to Chal rm n 

Curriculum Development Meetings 



19 
18 
16 
15 
12 
9 



25.0 
23.7 
2K1 
19.7 
15.8 
n .8 



Table 5 provides a summary of the chairmen responses to the question 
"What activities are conducted by your department that focus on the 
faculty member's Improvement of their own Instruction?" Very few chair- 
men at the University of Florida have established Instructional Improve- 
ment programs of any substance for their faculty. This finding suggests 
that there is e need for instructional development programs In mst 
departments. 
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Table 6 



Summary of Responses Regarding the Weighting of 
Items for Tenure Decisions 



Categories Chairmen Responses^ 



Teaching 28 

Research 33 

Se rv i ce ] ] 

Varies with V/orkload ]6 

No Response 3 

Other 2 



^More than one response possible 

Table 6 provides a summary of chairmen responses to the question: 
"Could you rank tbd following areas (l=Most Important) in terms of the 
weight they receive in this department in tenure awarding decisions?" 
It is interesting to note here that 28 chairmen placed teaching abiMty 
as the most important consideration in tenure decisions. This finding 
suggests the University of Florida may have a more positive climate for 
instructional innovations and change than is indicated by other data. 
Also the fact that a number of chairnen vary the criteria for awarding 
tenure suggests that chairmen are not --.eked into a reward system 
focused solely on an individual's research potential • 
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Classification of Departmental Chairman and Faculty as Being Open or 
Closed to Instructional Innovation and Change 

Data collected from departments was utilized to categorize depart- 
ment chairmen and their faculties in the following fashion: 

(1) Type One Departments (Open to Changing Instructional Programs). 

(a) Chairman Open to Change 

(b) Faculty Open to Change 

(2) Type Two Departments (Partially Open to Changing Instructional 
Programs) 

(a) Chairman Open to Change 

(b) Faculty Not Open to Change 

(3) Typo Three Departments (Partially Open to Changing Instructional 
Programs) 

(a) Chairman Not Open to Change 

(b) Faculty Open to Change 

(k) Type Four Departments (Closed to Changing Instructional Program) 

(a) Chairman Not Open to Change 

(b) Faculty Not Open Lo Change 

Classification of Chairmen and Departmental Faculties Using Attitude Scales 
Attitudinal scores were used to select two groups of faculty and chairmen: 
those who had very favorable attitudes toward the utilization of educational 
technology (ET) e.g., scores below one standard deviation from the mean; 
and those who had very unfavorable attitudes toward the utilization of 
educational technology (ET) , e.g., scores above one standard devI^^Ton 
from the mean. By selecting one standard deviation from the mean we 
would be dealing with approximately 32?; of the sample or about ]6% of 
the sample at each end of the continuum (favorable vs. unfavorable 
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attitudes). Table 7 indicates the descriptive statistics obtained 
from the faculty and chairmen. 



Table 7 

Descriptive Statistics For 
Faculty and Chairmen 





Respondent 


N 


Mean 


Standard 
Deviation 


Re U ability^ 


Chal rmen 
Facu' ty 


A55 


52.66 
A8.52 


12.82 
11.38 


.72 
.83 



^Represents Hoyt's reliability coefficient 



From the results In Table 7, it was found that 15 chairmen had 
very favorable attitudes toward the utilization of ET (one standard 
deviation below the mean), and 15 chairmen had very unfavorable 
attitudes toward the utilization of ET (one standard deviation above 
the mean). As expected ^h^se 30 chairmen represented hO% of the 
sample. Seventy-four faculty had very favorable attitudes coward 
the utilization of ET (one standard deviation below the mean), and 
80 faculty had very unfavorable attitudes toward the utilization of 
ET (one standard deviation above the mean). Again these 15^ faculty 
represented 3k% of the sample. 

Because the attitude scale had such high internal consistency for 
chairmen (.83) the results from it were used in conjunction with the 
interview questlonnal re data to make decisions concerning the 'bpenness** 
or "closeness** of departments surveyed on the University of Florida 
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The classification of department faculty groups as being either 
"open" or "closed" to change based on their attitudinal data was the 
next step. Here the data was incorr^Dlete , 21 department faculties were 
not surveyed. Not only was there a great deal of missinn data in this 
phase of the work but there was also a low response from many depart- 
mental faculties. 

It was decided that department faculties with 10% or more of their 
faculty scoring one or inore standard deviations below the faculty 
attitudinal mean score would be classified as 'bpen" faculties. Depart- 
ment faculty groups with ]0% or more of their faculty scoring one or 
more standard deviations above the faculty attitudinal mean score would 
be classified as "closed" faculties. 

Table 8 depicts the best classification of our sampled chairmen and 
departmental faculty groups that we were able to make using our con- 
ceptual scheme. It is clear to us that this is at best a very tenuous 
classification given the incompleteness of our data and the very low or 
no response rates of some of our department faculty groups. 

The classification of departments as they appear in Table 8 was 
achieved by a comparison of the data gathered. Some of the departments 
sampled fell clearly into our original proposed categorization syi;tem 
(see Table 8). However, because of the lack of faculty data or con- 
flicting information, we were unable to classify a large number of the 
sampled departments. Six departments were found to have chairrr^-sn and 
faculties who were open to change, four departments had open chairmen and 
closed faculty groups, and three departments had closed chairmen and 
faculty. None of the departments fell clearly into our Type Three Category 
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Table 8 



Classification of Sampled Departments According to 
Original Concept uai Scheme 



Type One Depts: Open-Open 



Dept. No. Dept. % Dept. Faculty 

No. Facul ty Surveyed 



10i» 28 71. 

105 8 87.5 

221 6 83.0 

301 9 88.9 

30A 29 16.2 

309 lA 100.0 

Type Two Depts: Open-Closed 

117 17 58.8 

301 8 88.9 

302 3<i ijij.o 
309 14 100.0 

Type Three Depts: Closed-Open 



Type Four Depts: Closed-Closed 

315 30 76.0 

317 15 76.0 

11 72.0 
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Concl us Ions 

Conclusions Related to the Research Study 

The results of the study Indicate that. In general, there tends 
to be a posltl/e climate for the utilization of Instructional tech- 
nology at the University of Florida, as measured by the perceptions 
of departmental chairmen. Secondly, It Is possible to make tentative 
selections of those departments which are most open to change and 
those which are most closed. Finally ll Is concluded that further 
work within specific departments should be undertaken to provide 
Information which will substantiate or fall to substantiate the 
study's categorizations. 

Conclusions Related to the Role of lnstitu^.;ol>al Research In Instruc- 
t i ona 1 Deve 1 opmen t 

Quite clearly, the research conducted In this study was Institu- 
tional In nature. It is the type of research that could be carried 
out by an Office of Institutional Research In a community college, 
k year private liberal arts college or university for the purpose of 
providing institutional data that wouK-l be useful in staff or Instruc- 
tional development programs. 

Obviously, It will be necessary for us to gather additional data 
from department chairmen and faculty members before we can fmpJement 
sound instructional development programs. However, we do feel that we 
have identified the first component of an emerging model that will link 
Institutional research to instructional, faculty, and organizational 
development programs. 

21i 
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The enrollmefit of first-time-in-college students at the Univer- 
sity of Florida, Florida State University, University of South Florida, 
and Florida Technological University is analyzed according to the 
county of high school origin of the students. A model is built based 
upon the probability for enrollment at each university of high school 
graduates who immediately continue some form of education after their 
graduation. For counties within 50 miles of each university, the 
probability of enrollment drops off with the logarithm of the dis- 
tance. Beyond 50 miles, distance has no significant effect on the 
probability. 

An assumption of constant relative probability allows the cal- 
culation of enrollments when certain probabilities are forced to 
change. Probabilities must be adjusted when alternatives for the 
high school graduates are increased or decreased. Given the stated 
assumptions, enrollments at the four universities in the study have 
little effect on each other. 
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Distribution of First-Time-in-College Students within the State Oriiv* 
ersity System 

Background 

The intent of this project was to be able to predict the distribu- 
tion of first-time-in-college (FTIC) students within the State Univ- 
ersity System of Florida on the basis of data at hand. It was hoped 
that the method would be able to account for ceilings on RIC students 
and that it be general enough to allow for the prediction of enrollments 
at campuses which do not currently exist. An important criterion for 
the procedure was that the analysis should be based only on 
"projectable" parameters. That is, the student distribution should be 
dependent only upon parameters which we can predict separately. Another 
major objective of the study was to gain insight from FTIC patterns for 
application to other enrollment patterns, e.g., to analyze the transi- 
tion from the community colleges to the State University System. 

Special interest in FTIC students results from two observations. 
First, currently there are only two policies which restrict enrollments 
within the State University System to specific numbers, year after 
year. One of these addresses ceilings for FTIC enrollment at each of 
the appropriate universities. Second, the enrollments of FTIC students 
form the first input into a student flow model for enrollments at all 
levels within a university or the System. A student flow model for the 
State University System would require an understanding of the FTIC 
variable. 

The study carried out by Carroll and Relies (1975) of the Rand 
Corporation (henceforth called the Rand study) investigated the use of 
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a number of variables for predicting the transition. Among other 
thing:^, it showed that ability as measured by the Florida Twelfth Grade 
Test and economic status had some bearing upon the decision of students 
to attend the University System. However, by the above criterion, 
student ability was omitted from direct consideration in the current 
study due to the difficulty of projecting average test scores or projecting 
other similar direct means of measurement. 

The Rand study showed that ethnic group was relevant only for 
attendance choice relative to Florida A&M University (FAMU), which has 
been predominantly black. The same study found that the actual total 
FTIC enrollment at each of the universities and the distance between the 
university and the high school of the prospective student were relevant 
variables for describing the attendance of any given student. Enroll- 
ments and distances are readily measurable, and it appeared that these 
variables should be included in the current analysis. 

It was expected that many variables could be accounted for by 
considering the percentage of high school graduates (by county) who 
continue their education after high school. "Education" here includes 
all post-secondary education, not just education within the State Univ- 
ersity System. The percentage of high school graduates who do not 
immediately continue their education ranges from about 90% to 20%. Use 
of the percentage as a proxy for other variables which would reflect the 
social norms, average student ability, etc., would be most useful if the 
variable was independent of the other parameters used in predicting the 
attendance within the State University System. This concern is ad- 
dressed below, in the section on the analysis of 1974 and 1975 enroll- 
ments. 21G 
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The analysis of previous years* patterns of attendance indicates 
that the patterns do change from year to year, apparently due to state 
economic conditions and other variables which are not constant in time. 
For example y between 1974 and 1975, when the number of high school 
graduates continuing increased by 6.0%, the Florida FTIC enrollment at 
University of Florida (UF), Florida State University (FSU), University 
of South Florida (USF), and Florida Technological University (FTU) 
increased only ^.8%. However, between 1973 and 1974, while the number 
of high school graduates continuing their education increased by only 
5.8%, the total Florida FTIC enrollment entering UF, FSU, and FTU increased 
^Z.3%. For this reason, the current study emphasized the relative 
distribution of students without great concentration upon the absolute 
magnitudes of the attendance. 

Analysis of 1974 and 1975 Enrollment 

It was decided to aggregate the potential students by county of 
high school origin, rather than by each high school of origin. The 
number of high school graduates continuing their education immediately 
after graduation is available yearly, by county, in the Department of 
Education publication "Florida High School Graduates." The choice of 
county rather than high school was based primarily upon the expectation 
that— relatively—the number of high school graduates by county would 
fluctuate less than the number of graduates by high school. The number 
of counties, 67, is large enough to provide "reasonable statistics." 
For the analysis, the universities accepting first-time-in-college 
students with the exception of FAMU were included. The four included 
are FTU, USF, FSU, and UF. The combination of the 67 counties with the 
four universities meant that there were 268 individual transitions to 
analyze. 
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The object was to predict the probability of high school graduates 
by county to attend each of the four given universities. By use of the 
percentage of graduates who immediately continue their education, the 
probabilities can be based on those high school graduates continuing. 
The parameters studied to explain the probability were the distance from 
the university (main campus) to the city of largest population within 
each county and the total first-time-in-college enrollment at the univer- 
sity. 

In other words, the dependent variable was p, the number from each 
county enrolling at a given university in the fall quarter divided by 
the number from the county graduating from high school the same year who 
immediately continued their education. The independent variables were 
D, distance, and F, total first-time-in-college enrollment. Many dif- 
ferent forms of the variables were attempted, for example, log p/(l-p)- 
However, the best fit using these parameters was found by a linear 
combination of log D and F to give p- (The situations where D = 0 were 
replaced by D = 10 miles.) 

One insight into the analysis of the problem came in breaking the 
transitions from county to university into two cases which appear to 
have separate types of behavior. The first case, called the "near 50" 
case, consisted of the 31 situations in which the distance from county 
to university was less than 50 miles. The other case, the "far 50" 
case, included the remaining 237 transitions in which the county contri- 
buted to a SUS university more than 50 miles distant. The distance, 
roughly speaking, is commuting range, (Limiting distances of 30 to 80 
miles were tried, and 50 optimized the F-statistic) 

The degree of a good fit (R^) was calculated on the basis of the 
actual number predicted to enroll in a university from each of the 
counties, not on the probabilities themselves. The "near 50" case was 



found to depend almost completely upon the distance between university 
and county, not the FTIC enrollment of the university. The "far 50" 
case showed just the reverse, almost complete dependence upon the FTIC 
enrollment and very little improvement in fit with the inclusion of the 
distance. The equations found for the probabilities for each of the two 
cases, in 1975, with the associated R^'s, are as follows: 

near 50: p = 0.64128 - 0.40650 log = 0.872 

far 50: p = -0.02635 + 0.00003U44 F, = 0.949 
It can be seen that the "far 50" equation explains almost all of 
the variation in the number of students from a county enrolling at a 
university. (Inclusion of the distance term increased R by less than 
0.0002.) While the "near 50" case does not have as high an R^, still 
the equation explains nearly 90% of the variation with the single 
variable, distance. Inclusion of the FTIC term would increase R by 
0.0163 but with a standard error in the coefficient of half the value of 
coefficient. Results for 1974 were similar, but showed somewhat more 
dependence on distance in the "far 50" case. 

The question may readily arise as to the selection of the first- 
time-in-college enrollment as an independent variable rather than some 
other enrollment or other number which would characterize the attractive- 
ness of the university to the individual high school student. One can 
maintain the same coefficients as in the above expression for the "far 
50" case, but require that the formulas give enrollments exactly 
for each of the four universities, and treat F as an unknown. Equi- 
valently, one can say that the probability for attendance is a constant 
dependent only upon the university; then the constant can be derived 
from the data to give an exact fit for the enrollments associated with 

219 



203 



the "far 50" case* That is, for the "far 50" case we can set p = A, 
where A = 0*05588 for th^ University of Florida, 0.03989 for Florida 
State University, •01863 for the University of South Florida, and 0.00390 
for Florida Technological University. A may be thought of as a long- 
distance attraction for the university, depending on such factors as 
space available and the university's reputation. 

The result of calculations based on the above fon^ulas is shown in 
Table 1. The first line is the actual 1975 enrollment from known Florida 
counties. The second line lists the enrol Irnents as calculated. 

Table 1 
Florida FTIC Enrollment 







FSU 


USF 


FTU 


Fall 1975 Actual 


2652 


2119 


1672 


863 


Calculated 


2519 


1917 


1502 


869 



We conclude the section on analysis by returning to the question of 
the stability of the percentage of high school graduates who immediately 
continue their education. We desire this percentage to be insensitive 
to the opening of additional universities or the imposing of restrictions 
on the enrollment of existing ones. If the percentages were sensitive 
to changing conditions, one would expect the percentages to depend upon 
the current relative locations of the counties with respect to the 
universities. This does not appear to be true. In the four counties 
which house the four universities under consideration, the overall 
percentage of high school graduates in 1974 who did not continue their 
education was 54.82%, whereas the state overall average was 49.21%. 
That is, in the precise counties in which one would expect low percentages 
of students not continuing if distance were a strong influence on the 
percentage, the percentage is higher than the state total. As a more 
sophisticated test, we calculated the correlation between percentage of 
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students not immediately continuing their education and log D (where the 
distance for the county was measured to the nearest university). The 
magnitude of the correlation was *063, low enough to believe that the 
percentages would not change significantly as the appropriate distances 
were changed* 

FTIC Di'stribution After Opening a New University 

Under the assumption of the validity of the formulas presented 
above and the assumption of "constant relative probability," to be 
described, one can predict the enrollments resulting from the initiation 
of a new university. As an example, one might hypothesize the opening 
of a lower-division SUS institution located in Miami. The probabilities 
derived above were calculated in the absence of the new institution. 
Hence all probabilities need to be adjusted to avoid requiring that the 
sum of the probabilities for all alternatives exceeds unity. 

After the probability for any given student to enter the new institu- 
tion changes from zero to a value greater than zero, the sum of the 
other probabilities for that student must become 1 less the new- 
institute probability. Mathematically, we let p^ be the former probability 
of the ith alternative, e.g., going out of the state, and p^^ be the new 
probability for the same alternative. In addition we let the probability 
of attending the new institution be Pq\ and sum starting with i = 1. 
Then: 

S Pi = 1 

We now assume that the probability of any alternative relative to any 
other alternative stays the same. That is, for any i and j, 
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P1/Pj = Pivpj^ 

This assumption (constant relative probability) leads to the expression, 
p\/p^ = 1 - Pq^ for i ^ 0. 
One might assume that the range of alternatives to which this 
equation applies should include, for consistency, attendance at the new 
institution. That is, 

P^/Po = 1 - P^ 

where Pq would be the probability calculated prior to making the 
adjustment. The results of some algebraic manipulation would show, 

P^ = Po/^l + Pq)- 
Although intuitively the value of p^^ should be somewhat smaller than 

pQ, it is hard to justify the last expression. It will be used only as 

a lower limit. 

It must be stressed that these results apply county by county. For 
example, the probability of entering FTU from Dade county, where the 
probability of attending the new institution is highest, will be reduced 
more than the probability of entering from Orange county. Consistent 
with intuition, it can be seen that opening a new institution will 
affect nearby universities (and colleges) more strongly than those far 
away. 

For this calculation, the long-distance attraction defined in the 
analysis above as A was assumed to be the same for the new Institution 
as for USF. The model predicted the Florida FTIC enrollment for the new 
university to be 3040, greater th^n at any of the other SUS 
universities. The lower limit on the prediction is 2602. Although FSU 
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and UF are calculated to drop by more than 100, they can be assumed to 
stay at their limits. However, the opening would have reduced USF by 69 
and FTU by 25. 

Effect of Enrollment Ceilings on the Distribution 

When policy limits the FTIC enrollment at given universities 
through designated ceilings, the result can be handled by the adjustment 
of probabilities. As enrollments are projected into futui-e years, in 
which the number of continuing high-school graduates increases, the 
probability of entering an institution at which policy limits the enroll- 
ment must decrease. For example, since the fall FTIC enrollment at UF 
must remain about constant in the years ahead, the probability of attend- 
ing UF from any county can be said to decrease according to the ratio of 
current enrollment divided by the enrollment which would be projected if 
there were no ceiling. 

The interesting question is, what happens to universities which are 
not at their ceilings as the probability for enrollment at other univer- 
sities is forced to decrease? To ask th^ question another way, to what 
extent are students deflected from one Institution to another? 

We use the assumption of constant relative probability and the 
reasoning developed for the opening of a new institution. The existence 
of the ceilings is said to impose a constraint. Again, we let p^ and 
p^-^ equal the probabilities by county for the ith alternative, respectively 
before the constraint and after it. If the alternative corresponding to 

i = 0 is constrained to a known value, then the algebra results in, 

1 1 
P i = Pi 1 - PC for 1 ^ c. 

1 - Po 

Again, probabilities are adjusted' more in those counties for which the 
constrained probability is high. 
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Currently, in the SUS, both UF and FSU are at their FTIC ceilings. 
So students in future years must be deflected from those schools to 
others • For each 100 students not added to UF and FSU due to the ceilings, 
the model predicts that 3.45 will be added to USF and 1.95 added to FTU. 
That is, about 95% of the students who can not enroll due to ceilings 
will not enroll at any university covered by this study. 

As an example of the effects of changing probabilities, the 
conditions of 1981 are used. Table 2 summarizes some results of 
computer calculations. It should be stressed that Table 2 does not 
constitute the complete FTIC enrollment projection for 1981. The 
only changes assumed between 1975 and 1981 are those stated 
explicitly. Column 2, based on uniform growth, assumes simply that the 
FTIC enrollment will grow at the rate of eligible students, state-wide. 
Those figures should be compared with the results of the model applied 
to the individual counties. Column 3 assumes "no deflection", i.e., no 
effect from the two universities at which limits are effective. If the 
deflection from those universities is included, 27 students are added to 
USF and 15 to FTU, as shown in column 4. 

Table 2 
Florida FTIC Enrollment 

(1) (2) (3) (4) 

1981 1981 
1981 Growth By Growth By 

1975 Uniform County.* County.* 

Calculated Growth.* No Deflection Deflection from Limited 

USF 1502 1726 1728 1755 (+27) 

FTU 869 998 957 972 (+15) 

*Growth based on 12th Grade Average Daily Membership projections of 

Florida Department of Education. 221 
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The FTIC projection procedure described by Thomas (1975) as used at 
USF compares most closely with the current calculations which led to 
column 2. Thomas made additional adjustments for changes expected in 
(1) high school graduation rate and (2) percentage of high school grad- 
uates attending USF, while he combined all counties in projections of 
state 12th grade enrollment. He projected an increase in Florida FTIC 
enrollment at USF of 40% by 1981 compared to 1975. The major differ- 
ences in his work to result in 40% versus 15% in the current study were, 
first, different 12th grade enrollment projections and, second, Thomas* 
extrapolation that the probability for enrollment at USF will Increase 
by about 17%. 

Summary 

The enrollment of Florida FTIC students at UF, FSU, USF, or FTU can 
be explained with a simple model based on the enrollment probability, 
calculated by county, applied to high school graduates who irranediately 
continue some form of education. Within 50 miles of a university the 
probability of enrollment drops off with the logarithm of the distance. 
Beyond 50 miles, distance has no significant effect on the probability. 

An assumption of constant relative probability allows the calcula- 
tion of enrollments when probabilities are expected to change. Probabil- 
ities should be adjusted by the addition of alternatives, e.g., the 
opening of a new institution, or the known reduction in the probability 
of one or more alternatives, e.g., the imposition of enrollment limita- 
tions. With the stated assumptions, enrollment patterns at the four 
universities of the SUS have little effect on each other. 
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Identification and Evaluation of Instruments In 
Institutional Research 

by 

Stephan A, Lewis 

As institutional researchers study the functions of their institutions» 
they usually rely upon data that Is or could be collected through the normal 
administrative and records keeping processes. There are, however, occasional 
questions or problems that cannot be. addressed using these data. When this 
occurs, the researcher turns to some other means of gathering data, and this in 
many cases takes the form of a special test or survey instrument. The researcher, 
therefore, needs to know what is available as well as what will be useful for his 
particular purpose. 

Researchers at large institutions with extensive research capabilities can 
design and develop instruments well suited to the local research usage. 
Researchers at institutions with limited capacities mustj, however, look beyond 
their own walls for help. The problem for the researcher is to identify those 
instruments which are readily available and to choose the one best suited for the 
purpose at hand. It is the purpose of this paper to aid in the identification and 
evaluation of these instruments. 

As used here, the phrase "readily available instruments" is intended to 
to include those instruments which are currently In print, designed to be used 
without need for local modification or adaptation, and accompanied by those kinds 
of materials and services generally unavailable at small to medium sized 
Institutions and necessary for the proper utilization of the instrument. 
Instruments fitting this description are designed to measure several aspects of 
postsecondary Institutions: ooo 
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Institutional Climate or Environment 

Surveys of institutional climate or environment take either or both of two 
approaches to the measurement of the characteristics of the Institutions 's 
students, faculty, administration, and physical plant which, when taken in 
combination, give the institution its uniqueness. The first of these approaches 
measures how the respondents (students, faculty, etc.) perceive qualities or 
subjective characteristics of the institution's environment. The Instrument 
(the College and University Environmental Scales , for example) might ask the 
respondent to agree or disagree with the statement: "Students set high standards 
of achievement for themselves." These subjective observations or perceptions 
are usually grouped to yield scores for the institution on several scales. Each 
scale is intended to represent a dimension of an institution's environment; 
the score on a scale indicates that the institution is characterized by more or 
less of the attribute or group of related attributes represented by that dimension 
or scale. 

The second approach, as In the Inventory of College Activities , arrives at 
the same type of Information, but begins by asking respondents to report the 
occurance of certain objectively observable stimuli (events, regulations, 
facilities, etc.) that are supposed to be related to or to have an effect upon 
the institutional environment. Again, responses are grouped to yield scores on 
scales intended to measure the relative amount^ of attributes that characterize 
an institution. Whatever approach is used, these instruments yield scores for 
Institutions only. Individual respondent's scores cannot be obtained. 
Surveys of Instructional Effectiveness 

Surveys of instructional effectiveness also make use of either or both of 
these two approaches. In the one case, respondents, usually students, are asked 
for their subjective judgements on the amount of various attributes of instruction 
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that existed in a particular course. The other approach, as in the Instructional 
Development and Effectiveness Assessment System, asks respondents to report the 
occurance of events that are supposed to be related to the quality of the : : 
instruction in a particular course. Again, as in the measures of environment, 
items on the instructional effectiveness measures are usually grouped to provide 
scores on several scales, intended to represent dimensions of instruction; scores 
are obtained only for individual course sections, the measures cannot yield 
scores for individual respondents. 
Vocational or Occupational Interest Surveys 

Vocational or occupational interest surveys av^e designed to yeild scores 
for individual respondents and are intended to be used primarily as tools to aid 
in the guidance and counseling of individuals such as students. Complilations 
of individuals' scores from measures of this type can yield useful information 
about the interests of the group to which the instrument is administered. 
Individuals responding to surveys such as the Kuder Occupational Interest Survey 
are asked to express preferences for a large variety of work activities that can 
be related to various occupational groupings. Alternatively, they may be asked 
to indicate their preferences from a variety of occupational titles, as in the 
Strong^Campbell Interest Inventory . In a manner similar to the two previously 
mentioned types of instruments, an individual's responses to items are grouped 
to yield scores on scales that are intended to represent the major dimensions of 
the world of occupations. 
Surveys of Demographic and Historical Data 

There are several questionnaires or surveys designed to gather demographic 
and historical data about the collective student body. These instruments may 
also survey students' opinions and attitudes about issues not specific to the 
institution. The attractiveness of these commercially available instruments is 
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that local results may be compared to norms derived from nationwide administrations 
of the same instrument. Here again, scores may not be yielded for individual 
students. 

General Achievement or Ability 

Measures of general acheivement or ability are available for administration 
to students at all levels of postsecondary education, freshman through graduate. 
These instruments generally are designed to yield scores for individual respondents 
on scales intended to be representative-of general verbal and/or quantitative 
skills. Scores for individuals can usually be compiled to produce information 
about the general ability of the group tested, and some instruments are supplied 
v/ith norms for the purposes of reference. Unfortunately, there does not appear 
to be any one instrument that would be suitable for a pretest-posttest type 
experiment over the two or four year educational experience. 
Specific Abilities or Achievement 

Instruments designed to measure specific abilities or achievement are legion. 
The use of this type of measure is slightly removed from the central thrust of 
institutional research and they are therefore not considered in this presentation. 
Attention is directed to the extensive lists of instruments contained in the 
volume sponsored by the Council on Postsecondary Accreditation: Evaluation of 
Institutions of Postsecondary Education; An Annotated List of Instruments , and to 
Oscar Krisen Buros' Tests in Print II . 
Multiple Purpose Packages 

Finally, The American College Testing Program (ACT) and the Educational 
Testing Service (ETS) offer packages of instruments designed particularly for 
two year postsecondary institutions. These packages include instruments designed 
to aid in the collection of data about students' abilities and interests as well 
as demographic and historical information. Although these packages are designed 
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primarily for guidance purposes, the analytical services associated with their 
use may make them useful for institutional research. 
Evaluation and Selection of Instruments 

Once the instruments that might suit his purpose are identified, the 
researcher must procede to evaluate those that best fit the restrictions of the 
given situation. Experts in measurement generally divide the criterea for the 
evaluation of an instrument into three categories: reliability, validity, and 
practicality. The relevant questions here is: what do these terms mean to a 
practicing institutional researcher? 
Reliability 

Reliability is the most commonly mentioned and should be addressed first. 
More importantly, it is the primary criteria, for the reliability of an instrument 
places an upper limit on its quality and worth. If it is not sufficiently reliable 
it is of little use no matter what its other qualities may be. 

Institutional researchers use reliability to measure the stability of scores 
yielded by the measure. Instrument publishers should report reliability in the 
technical materials that are made available to the prospective user, and they 
should report their instruments* computed reliabilities as well as the nature of 
the group whose responses were used in the relibility studies. The publisher may 
offer indices of reliability computed for individual or group scores. If there 
is not reason to believe that some bias will exist within the local sample that 
would produce a constant error, group scores can be expected to have a higher 
reliability than that indicated for individual scores. In any case, the indices 
provided by the publisher are only meaningful to the local user if the local 
group of respondents is sufficiently similar to the froup or groups used in 
developing the instrument and computing its reported reliability coefficients. 

For the researcher who wished to employ a published instrument, sufficient 
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little administrative or analytical effort on the part of the researcher; 
another might be the opposite, inexpensive to purchase but requiring many hours 
for administration, scoring, and analysis. 
The Instrumen ts 

With all these criteria in mind, the following list of instruments vias 
compiled. The list is not intended co be definitive or exhaustive; it is intended 
to provide practicing institutional researchers a useful and ready reference to 
help identify currently available instruments. Several sburces were helpful In 
compiling the list. They included Oscar Krisen Buros* Tests in Print II . the 
volume sponsored by the Council on Postsecondary Accreditation, the Test Collection 
Bulletin , and the Annotated Bibliographies of the Association for Institutional 
Research. Additionally, Dr. G. Emerson Tully of the Florida Board of Regents 
Staff sent a letter to many of the institutional researchers in the state of 
Florida, asking for lists of instruments with which they were familiar. Responses 
to this inquiry, combined with the search of the references mentioned, helped the 
author identify a large number of possible instruments. 

From the list of possible instruments thus compiled, the author made an 
effort to select those most appropriate for institutional research and those that 
best fit the definition of •Veadily available instruments". It is hoped that this 
list will prove to be of value and use to those busy institutional researchers 
for whom it was prepared. 
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SELECTED INSTRUMENTS 



Measures of Environment: 

The College urd University Environment Scale^ Second Ed1t1on# (CUESII) 
institutional Research Program for higher Education, Educational 
Testing Service, (IRPHE, ETS) 

Developed by C. Robert Pace for ETS from the old College 
Characteristics Index, the CUESII measures students* perceptions 
of several aspects of the institutional climate or environntent. 
The technical manual presents extensive evidence of reliability 
and construct validity, CUESII is not considered to be 
appropriate for use with freshmen or first term sophmores. ETS 
no longer offers scoring and reporting services; scoring may be 
accomplished locally. Booklets are $.35 each; answer sheets 
$.05 each. 



Higher Education Measurement Evaluation Kit , (HEMEK) 

C. Robert Pace and the staff of the Higher Education Evaluation 
Program, Center for the Study of Evaluation, Graduate School of 
Education, University of California - Los Angeles. 

For the institution willing to invest more labor than money, 
this kit offers great flexibility. It contains a large number 
of mini-scales, allowing for measurement of student perceptions, 
and for student reporting of the incidence of Important 
environmental stimuli. For some of the mini-scales, testing 
has not been accomplished nor are norms available. For those 
scales tested, norming was done with college and university 
upperclassmen; use at two«year institutions may not be appropriate. 
Reported reliabilities range from fair to good. Use of the kit 
requires local choice of scales, reproduction of response sheets 
or booklets, scoring and analysis. The kit is free for the 
asking. 



Institutional Functioning Inventory. (IFI) 

Institutional Research Program for Higher Education, Educational 
Testing Service. (IRPHE, ETS) 

The IFI is designed to measure institutional vitality, institutional 
goals and commitment to them as perceived by various constituent' 
groups such as faculty, administrators, students and others. The 
technical manual contains extensive information indicating high 
reliability and construct validity on all eleven scales. 
Approximate costs of ETS supplied materials, scoring and reporting 
are $1.00 to $1.25 per respondent. 
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Institutional Goals Inventory, (IGI) 

Instlrutional Research Program for Higher Education, Educational 
Testing Service. (IRPHE, ETS) 

The IGI is an Inventory or checklist of 90 Institutional goal 
statements. Respondents (faculty, $tudc;its or other constituent 
groups) indicate the degree to which tney perceive each stated 
goal as an actual goal of the Institution, and the degree to 
which they believe the stated goal should be an actual goal of 
the Instltf ".Ion, Information on reliability and validity Is not 
available. Costs are $1,60 per respondent for materials, scoring 
and reporting. 



Inventory of Corege Activities, (ICA) 
Natlonal Coriiputer Systems. (NCS) 

As designed by Alexander H, Ast1n» the ICA measures environment 
based upon environmental characteristics or stimuli that may be 
readily observed and reported by students, and which seem likely 
to have some effect upon student development. Evidence on 
reliability and validity presented in the ICA Manual should be 
carefully weighed. The ICA may be locally scored, NCS offers 
scoring and reporting for approximately $,70 per student respondent. 



Student Reactions to College Questionnaire, (SRC) 

Institutional Research Program for Higher Education, E'iucational 
Testing Service, (IRPHE, ETS) 

Designed especially for use at two year Institutions, the SRC 
is intended to measure students perceived concerns and how well 
those concerns are being met. Results are reported on all itemsj 
items are not grouped into scales. The users manual contains a 
short discussion on Item reliability. Discussion of construct 
validity is not appropriate. Cost is $1.60 per student respondent 
for materials, scoring and reporting. 



College Student Questionnaires. (CSQ) 

institutional Research Program for Higher Education, Educational 
Testing Service. (IRPHE, ETS) 

The CSQ are three instruments: 

Part I, intended for administration to incoming freshmen prior 

to the opening of classes, is designed to elicit demographic 

information and information about secondary educational experiences, 

family background, and student attitudes. 

Part II, intended to be administrated to any group of students 

toward the end of the school year, emphasizes student attitudes 

and perceptions of institutional activities and functions, 

A short Control Test for Academic Aptitude (CTAA) may be used In 

conjunction with either Part x or Fart II. 

Extensive evidence of scale reliability and construct validity Is 
contained in the technical manual. The CTAA Is onlv sufficiently 
reliable for use as a group measure. Booklets are $.35 apiece 
for each of Parts I and II j CTAA booklets are $,15 each; answer 
sheets are $,05 each. The CSQ must be scored locally; ETS scoring 
and reporting are no longer available, 

221 




235 



Measures of Instructional Effectiveness: 



Instructional Deyelopcnent and Effectiveness Assessment System* (IDEA) 
Center for Faculty Evaluation and Development In Higher Education. 

In addition to student perceptions of instructional effectiveness » 
this Instrument is designed to measure characteristics of 
instruction as observed and reported by students. Reviews of this 
Instrument are not yet available* Costs, on a sliding scale, 
begin at $.30 per student respondent for materials, scoring and 
reporting, 

minois Course Evaluation Questionnaire* (ICEQ) 

Measurement and Research uivlslon, Universit;y of Illl^jols. 

The ICEQ is designed to measure student attitudes toward the 
instructor and the instructional environment. Results reported 
by the test authors indicate good reliability and validity. Costs 
are $.15 per student response for materialSo scoring and reporting. 



Report of Course Activities, Attitudes and Characteristics. 

C. Robert Pace, Higher Education Laboratory, University of California - 
Los Angeles. 

This Is a series of questionnaires to gather data from students 
and Instructors concerning students' perceptions of instructional 
quality, faculty teaching preferences, and students' learning 
style preferences. Information on reliability and validity Is 
not available. Single copies of the instrument are free, 
reproduction, scoring and reporting are intended to be accomplished 
locally. 



Student Instructional Rating System. (SIRS) 

Office of Evaluation Services, Florida State University. 

This Instrument, adapted from the instrument of the same name at 
Michigan State University, is designed to measure students' 
perceptions of the learning experience in the evaluated course. 
Research indicates that the instrument is very reliable and is 
very stable along its various scales* Materials, scoring, and 
very extensive reporting services are available on a cost 
recovery basis. 

Student Instructional Report. (SIR) 

Institutional Research Program for Higher Education, Educational 
Testing Service. (IRPHE, ETS) 

This Instrument was developed by John A. Centra from the 
Michigan State University Student Instructional Rating System. 
It Is Intended to measure students' perceptions of various 
aspects of Instruction. SIR Report #3 contains an extensive 
discussion of Item reliability and scale validity. Costs for 
purchase of forms and scoring and reporting start at $.35 per 
student response and decrease with Increased usage. 
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Vocational Interest Measures: 



Kuder Occupational Interest Survey. (KOIS) 
Science Research Associates. Inc. 

Also known as the Kuder DD> this instrument may be slightly 
more suitable than the SCII for use at Institutions with 
vocational -technical emphasis. Reviews published shortly after 
the most recent revision In 1970, Indicate some shortage of 
reliability and validity data. None the less, much research has 
been done using this Instrument. The KOIS and the SCII are the 
two outstanding measures In the area of vocational Interest; 
reported research Indicates however that the two Instruments 
do not yield correctable results, and are therefore not 
equivalent. Costs are approximately $1.25 per student respondent 
for materials, scoring and reporting. 

Strong-Campbell Interest Inventory. (SCII) 
Consulting Psychologists Press, Inc. 

The SCII Is the latest In a long series of very widely used 
vocational interest Inventories/ Published In 1974, the new 
sen combines previously separate forms for males and females. 
Early research Indicates that the SCII will have high reliability 
and validity as did Its predicessors. The SCII makes use of 
occupational titles and emphasizes traditionally college level 
occupations; It may therefore be more suitable for institutions 
with an emphasis on more traditional college curriculum. 



Vocational Preference Inventory. (VPI) 
Consulting Psychologists Press, Inc. 

Designed primarily as a locally scored personality inventory, the 
VPI, also known as the Holland VPI, has been used more often as 
an Interest measure. Although reviews indicate that reliability 
and validity may be no more than adequate, the VPI's strongest 
features are its briefness and low cost. Booklets are approximately 
$0.06 apiece, answer sheets $0.07 apiece. 



Sources of Information About the Student Bo4y: 

Student Information Form. 

Alexander As tin. Cooperative Institutional Research Program, Graduate 
School of Education J University of California - Los Angeles. 

This instrument is administered to first time first term freshmen 
at a very large number of postsecondary Institutions throughout 
the country. The Information elicited Includes biographic and 
demographic data and information about high school background, 
career and educational aspirations, and need for assistance. The 
form Is altered each year to better sample attitudes toward 
current issues. Costs are $.40 per student respondent. Reporting 
includes national and local norms. 
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The College Entrance Examination Board offers reports to participating 

institutions based upon the Student Descriptive Questionnaire that is 
part of the CEEB Admissions Testing Program. The information covered 
includes biographical data, Information about high school activities 
and performance, future ^jducational plans, and needs for assistance 
and results of self -rating in several ability areas. Reports are 
available for those students who enrolled in the institution and for 
those who designated the institution to receive testing results but 
did not enroll. 



The American College Testing Program offers several services associated 
with its Student Assessment Program; 

Class Profile Service , 

Participating institutions may receive compilations of informati 
gathered through the Student Profile section. High School Grades 
section, and the Interest Inventory that are parts of the ACT 
Student Assessment Program. Compilations are available for 
those students who enrolled in the institution and for those 
students who designated the institution to receive assessment 
results but did not enroll. There is no charge for this service 

Institutional Self -Study Service. 

Participating institutions may receive reports based upon data 
from the ACT Assessment Program and from the Self -Study Service 
instrument (including standard plus special local interest 
items) administered to the freshman class at the end of the 
first term. This service is available at cost. 

The Basic Research Service and The Standard Research Service. 

These two alternative services offer predictive statistics and 
information based upon high school data reported by students 
through the ACT Assessment Program in combination with students' 
first term grades reported by participating institutions. There 
is no charge for this service. 

Measures of General Ability or Achievement: 

College Qualification Test. (CQT) 
Psychological Corporation. 

The CQT is a battery of tests measuring mathematical skills, 
vocabulary and knowledge of general information. Data indicates 
that the battery has high reliability and good validity for 
selective admissions and predictive purposes. The tests may 
however, be dated with the most recent edition prepared in 1960. 
Booklets cost approximately $.25 each, answer sheets $.08 each. 
The tests may be scored locally. 
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Culture Fair Intelligence Test, Scale 3> 

Institute for Personality and Ability Testing. 

This instrument is intended to measure, through the use of both 
universally familiar and universally unfamiliar material, latent 
potential rather than learned skills. This goal has been only 
partially accomplished, as is indicated by only fair reliability 
and validity data. Scale 3 is the highest level form available 
in the Culture Fair Intelligence Test Series, and is probably 
suitable for use with undergraduate students. Costs are 
approximately $.18 apiece for booklets and $.06 each for answer 
sheets. The test is intended to be scored locally. 



The Educational Skills Test, College Edition. (EST/CE) 
CTB/McGraw-Hill. 

Designed to assess the English and mathematical skills of 
entering freshmen at open-door colleges,' the EST/CE battery 
measures the abilities and achievement that students bring 
with them to postsecondary institutions. Information on 
reliability and validity Is not available. The battery may be 
hand or machine scored locally or scored by tine publisher. 
Costs are approximately $.25 and $.17 apiece for the English 
and Mathematics booklets, and $.06 apiece for answer sheets. 



Miller Analogies Test. 

Psychological Corporation. 

This test uses 100 verbal analogies to measure scholastic 
aptitude for graduate school. Rev;c . 'indicate that the 
instrument is very reliable; there is however, some question 
concerning the extent of its validity. The test may be hand or 
machine scored, and cost are determined by the local licensed 
administering center. 



Graduate Record Examinations Aptitude Test. (GREAT) 
Educational Testing Service. 

The GREAT is the first part of a program designed to aid in 
graduate school admissions selections. The test measures the 
general verbal and mathematical abilities of college seniors 
with considerable reliability. Normally administered nationally 
at established centers, six times a year, graduate schools may 
apply for local and convenient administrations. Costs are 
$10.50 per stu'ient. 
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School and CoHegFi Ability Tests, Series U. 

Cooperative Tests and Services, Educational Testing Service. 

The SCAT II is a series of tests available in alternate forms 
and designed primarily to predict academic success at the next 
highest scholastic level. The highest level for which the 
measures are suitable is the freshmen-sophmore level. Data 
indicates good reliability and fair validity. One strong feature 
is the short time needed for administration (approximately 40 
minutes). The tests may be scored locally. Costs are $.35 
for booklets, $.06 for answer sheets; ETS scoring Is $»40 per 
student. 



Undergraduate Assessment Program. 

Educational Testing Service. (ETS) 

This program, which Is not yet available, will be aimed primarily 
at the second term sophmore level, and will take the place of 
ETS*s Undergraduate Program for Counseling and Evaluation which 
was formerly the Undergraduate Record Examination. 



Miscellaneous: 

Career Planning Program. (CPP) 

American College Testing Program. (ACT) 

The CPP consists of a battery of Interest, ability and background 
measures, designed especially for use at postsecondary vocational- 
technical emphasis Institutions. Reviews indicate fair validity 
and reliability. The program is primarily oriented toward 
guidance but there are several research services associated with 
the program. Cost Is $6.00 per student for materials, scoring and 
reporting. The test Is administered locally. 



Comparative Guidance and PI acefnent Program. (CGP) « 

College Entrance Examination Board, Educational Testing Service. 
This program consists of a battery of background. Interest, 
general ability and special ability measures, oriented toward 
guidance of students admitted to two-year and vocational-technical 
institutions. The battery may be used In whole or in part, and 
several research reporting services are available. Reviews 
Indicate that the program's various scales have good reliability 
and fair validity. The test Is administered locally, and costs 
are $4.50 per student for materials, scoring and reporting. 



Faculty Activity and Outcome Survey Instrument. 

National Center for Higher Education Management Systems at Western 

Interstate Commission for Higher Education. 

This questionnaire form Is designed to aid In bringing together 
data on the amount of time faculty members devote to each of 
various perceived activities and the budgetary accounts that 
support each activity. 
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SELECTED REFERENCES IN ORDER OF USEFULNESS TO AUTHOR 



Tests in Print IIj^ An Index to Jests, Test Reviews, and the Literature 
on Specific lests . Edited by Oscar Krisen Buros^ The Gryphon Press 
1 ^74, (TIPII) 

The Seventh Mental Measurements Yearbook , Edited by Oscar Krisen Buros. 

The Gryph6n Press, 1972. (HMY7) 

These two publications edited by Euros are the best of all 
possible sources for information about testing and measurement 
instruments conmercially available through approximately 1973.* 



Evaluation of Institutions of Postsecondary Education; An Annotated List 
of Instruments , Regional Accrediting Cowmissions of Postsecondary 
tducatlon under sponsorship of the Council on Postsecondary 
Accreditation, May 1975, 

An extensive although not complete list of instruments is 
contained in this publication. The criteria for inclusion or 
exclusion are not clear. Some instruments listed appear to be 
only marginally suited for institutional research. Particular 
attention is paid to specific area tests of achievement. 



Test Collection Bulletin, published four times yearly by the Educational 
Testing Service, Princeton, New Jersey, 

This periodical publication lists all additions to the Test 
Collection maintained by ETS under the sponsorship of ERIC, 
The publication also announces test reviews, instrument 
discontinuations, and relevant address changes. 



Outcome Measures and Procedures Manual, Field Review Edition^ Technical 
Report 70 , Sidney S, Micek, Allen L, Service and Yong sl Lee. 
National Center for Higher Education Management Systems at Western 
Interstate Commission for Higher Education, May 1975, 

Containing an extensive list of higher education outcomes and 
suggested questionnaire items or mini-*instruinents and some 
references to cocnnercially available instruments for the 
measurement of the listed outcomes, this publication is the 
closest thing yet to an institutional research cookbook. 



Handbook of Academic Evaluation, Paul L, Dressel, Jossey-Bass Publishers 

m^. 

The discussion on the strengths and weaknesses of the use of 
measurement instruments that appears in the chapter on the 
measurement of environment is of particular interest. 
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Addresses 



American College Testing Program 
P.O. Box 168 
Iowa City, Iowa 52240 

Center for Faculty Evaluation and 
Development in Hiqher Education 
Kansas State University 
Manhattan, Kansas 66506 

College and University Programs 
Educational Testing Service 
Box 2813 

Princeton, New Jersey 08540 

College Entrance Examination Board 
888 Seventh Avenue 
New York, New York 10019 

Consulting Psychologists Press, Inc. 
577 College Avenue 
Palo Alto, California 94306 

CTB/McGraw-Hill 
Del Monte Research Park 
Monterey, California 93940 

Graduate School of Education 
University of California - Los Angeles 
Los Angeles, California 90024 

Institute for Personality and Ability Testing 
1602 Coronado Drive 
Champaign, Illinois 61820 

Measurement and Research Division 
University of Illinois 
307 Engineering Hall 
Urbana, Illinois 61801 

National Computer Systems 
Interpretive Scoring Systems 
4401 West 76th Street 
Minneapolis, Minnesota 55435 

National Center for Higher Education 

Management Systems at 
Western Interstate Commission for 

Higher Education 
P.O. Drawer P 
Boulder, Colorado 80302 



Office of Evaluation Services 
54 DIN 

Florida State University 
Tallahassee, Florida 32306 

Psychological Corporation 
Testing Division 
304 East 45th Street 
New York, New York 10017 

Science Research Associates, Inc. 
259 East Erie Street 
Chicago, Illinois 60611 
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REGIONAL LEARNING NEEDS ASSESSMENT: 
PRACTICES, PITFALLS, AND PROSPECTS 



I. Bruce Hamilton 
Educational Testing Service 



Presented at the Florida Invitational 
Conference on Institutional Research 

Orlando, Florida 
June 17-18, 1976 
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Regional learning needs assessment: Practices, Pitfalls, and Prospects 



!• Bruce Hamilton Ph. 
Assistant Director, Office of New Degree Programs 
Educational Testing Service 

For a variety of reasons, colleges are increasingly interested in determining 
what educational services are needed to meet the growing demand for learning 
opportunities from working adults, part-time learners, . and those who for various 
reasons cannot partake of traditional, on-campus educational offerings. Insti- 
tutional researchers have for several years been aware of the impending dearth 
of available graduating high school seniors over the next fifteen yoprs,"^ and 
the statistics which show the rapid growth in numbers and percentage in the 

population of those who are 25 to 44 years old— the adult group who have been 

2 

found to be the most interested in further education. Hut these officials 
are often uncertain as to what this means for their own college, how to plan 
for these changes, and* what new types of educational delivery or student 
services may be needed. 

In this circumstance the usefulness of field studies to reveal adult 
and noutraditional student educational plans, needs, and interests has gained 
in popularity. It is the primary purpose of this paper to briefly refer to 
several of these studies, review the utility of one survey strategy and 
methodology or another for particular purposes, and give an assessment of 
the prospects for better and more useful studies. It is hoped that this 
exercise will be of interest and material assistance to institutional 
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researchers and other officials whose responsibility it is to care for the 
educational needs of the future. 

The ri is ing popularity of learning needs surveys 

A'lult education has been viewed as a peripheral activity of the educational 
enterprise with a major purpose in bringing the adult population who were 
foreign-born or not well served in their youth up to a functional level of 
competence. Two early national surveys of adult participation^ rates and 
describing characteristics of adult learners were a U.S. Office of Education 
study in 1959, Participation in Adult Education , and another conducted by 
the National Opinion Research Center ;in 1962 and reported by John Johnstone 
and Ramon Rivera in Volunteers for Learning (1965).^ These early surveys 
provide the benchmark from which subsequent studies measure progress!. 

I 

In the 1960s and 1970s adult education became somewhat submerged nationally 
in the nontraditional study movement, which expanded the concept of adult 
education to one that asserts that anyone who wants and needs . further education 
has the right and ought to have access to an appropriate educational experience. 
The dramatic growth of the community colleges is one indication of this open 
learning philosophy, followed by the establishment of a host of experimental 
external and community-based degree programs monitored by the Office of New 
Degree Programs.^ 

Tlie national counting process continues. The National Center for Educational 
Statistics began a series of triennial surveys in 1969, which confirm the ' 
rapid growth of adult education.^ These surveys demonstrate that adult 
enrollment has increased by close to 15 percent each year since the mid-1960s. 
In 1972 the Educational Testing Service (ETS) contracted for a national 
survey of the potential market for continuing education with the Response 
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AnniysLS Corporation. This study was based on a national probability sample 
of 2,515 households, from which 2,004 responses from 1,248 households were 
generated. Tlie sample was then adjusted by several demographic factors 
to produce a final, weighted sample of 3,910 that was judged to be 
representative of the total U.S. population. This study estimated that 32 
million adult Americans were engaged in some sort of postsecondary learning, 
and that millions more were potential new learners if appropriate services 
were available. 

Tliese national surveys set the pattern for a host of state, regional, 
and local institution studies of learning participation and needs assess- 
ment for a variety of purposes. The Office of New Degree Programs has 

evidence that there were at least 47 studies completed between 1970 and the 

i 

present* In addition, at least 17 studies are currently under way, including 

statewide studies in Connecticut, Florida, Indiana, and Pennsylvania which 

g 

will be available before the year is out. By any measure, it is clear that 
field studies are a popular and useful means of forecasting demand and 
designing appropriate programs to meet it. The key question for planners 
is, what kind of study do we need, for what purpose, and who should do it? 
The answer to these questions lies in a careful look at the efficacy of 
various survey methodologies used in differing settings. 

Types of field surveys 

There are four basic types of field surveys used in these studies. 
By far the most common is an ordinary mail questionnaire. Mail surveys 
have the advantage of being a relatively inexpensive way of gathering data. 
If the population to be queried is well defined and of limited size, if no 
particular biasing effect would likely occur by the self-selection of responden 
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and if the rate of return and and absolute nur.iber of usable returns relative 
to the universe population is adequate, then it is a satisfactory method to 
use. A number of random sampling and respondent selection techniques may need 
to be employed to avoid serious errors in interpretation. Even then, serious 
sampling errors can occur: response rates may be too low, self-selection 
biasing effects can be severe, and total sample size in relation to the 
universe may be too small and unrepresentative to provide meaningful results. 

One must remember that in the mail survey technique, respondents 
are doing the selecting, not the researcher, no matter how carefully one 
sets up the strategy. In a. study of a region where 10,000 adults reside, 
and 500 questionnaires are randomly mailed to residents using the telephone 

directory, very little can be scientifically inferred from a response of 

i 

150 questionnaires. The response rate, the total number of responses (1.5 
percent of the universe), the biasing effect of a directory selection technique 
respondent self-selection, and unknown sampling errors add up to a meaningless 
effort. Mail surveys are not generally satisfactory in large-scale sampling 
to determine characteristrlcs of the universe population. 

The same cautions apply to other wide-scattered, unfocused sampling 
techniques such as newspaper cut-out questionnaires, surveys left in super- 
markets and other public places, requests for data using broadcast media, 
and the like. 

A second general Purvey type is a telephone survey, very popular with 
polling firms for its relatively Inexpensive, rapid-result, first-hand contact 
characteristics. Probably the majority of surveys that call for sample sizes 
of less than 1,000 are now conducted by telephone. 

A major advantage to this method is the control this strategy allows 
the researcher in selecting respondents. By carefully analyzing the population 
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characteristics of a given region, and selecting respondents by a variety of 
methods so as to accurately reflect the universe population, studies with a 
very small size can yield meaningful data about the total population. The 
methods used in these screening and selecting activities are extremely 
technical, and novices should exercise care in employing them without the 
guidance of an experienced pollster; but the power one gains through telephone 
surveying methodology is substantial. An accompanying effort must be included 
to control for that fraction of the population (often the poor and the 
transient) not available by ordinary telephone contact, and weighting the 
sample accordingly. , 

The speed and accessibility afforded by telephone contacting allows 

for another advantage: filtering. A narrow band of the population who 

1 

share a common characteristic wanted in a survey can be systematically * 
revealed by filtering out respondents in a larger sample who report that they 
lack the characteristic. For example, if females with at least two years 
of college but less than four years, who are working or have children at 
homc?^, are the respondents required for a particular survey, a sampling design 
may call for telephone contacts with up to 1,000 households before a subsample 
of 100 persons having these characteristics is found. The 100 are then asked 
the enitire survey, and the remaining 900 contacts can be eliminated after 
only a brief exchange on the telephone. Telephone appointments .can be made 
with contacts who may not have time available at the hour of first approach, 
Increasing tlie rate of response to a high level. 

Tiiere are, however, certain disadvantages to telephone surveys. One 
is the limitation to a short set of questions that can be answered in simple 
replies. Also, it is easy to misinterpret certain replies over the telephone, 
leading to some over-or-under estimation of responses to difficult questions. 
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Telephone contacts that last beyond a few minutes begin to break clown unless 
a specific appointment is made for a longer interview. 

Another disadvantage to telephone surveys over a wide area is the cost 
that may be involved, and since the sample size is likely to be quite small 
in terms of the universe, it is easy to bias the results through sampling errors 
in the methodology of respondent selection or through faulty weighting of 
the data to conform to population characteristics. In certain areas of the 
countrv language problems can be a barrier to effective communication. 

Finally, certain types of studies require respondents to choose from 
n range of choices preconceived by the researcher and presented to 
respondents in a written format. The deeper the interview, requiring careful 

weighing of conflicting feelings and motives, the less effective the telephone 

i 

survey technique. ' ' 

Interview polling by face-to-face question and answer is perhaps the 
best known method of gathering statistics about a sample of the population. 
The advantages are clear: in-depth questions are possible through face-to- 
face exchange of cards of printed response choices, and respondents are often 
more honest and careful in their replies when talking to a person than is 
usually the case in more remote and impersonal survey techniques. Moreover, 
lengtliy surveys can be conducted by interview appointment, given lead tim6 
and resolute interviewers who are free to return to a sample household on 
several occasions. 

A major disadvantage is the high cost of this method. Home-appointment 
interviews are the most expensive way to collect data, for interviewers* 
time must be covered, and travel to the selected points of contact, training 
expenses for interviewers, the cost of carefully selecting the towns, 

249 

235 

O 

ERIC 



neighborhoods, and blocks to produce a sample conforming as closely as po55sible 
with the characteristics of the universe population all need to be paid for. 
Moreover, if the population contains severial distinct ethnic groups, interviewers 
need to be carefully chosen and trained to reduce the social barriers that 
may limit the effectiveness of the interview technique. 

Some educational researchers have found that one way of reducing the 
cost of an interview survey is to "piggy back" questions onto another survey 
that has been planned by another group, or to take advantage of a regular 
survey service that has flat per-question rates to customers who buy into 
the design offered by the service. Care must be exercised in these situations, 
however, since whether one "piggy backs" or purchases services, the sample 

size and survey design are not one^s own, and may be inappropriate for the 

I 

educational survey purpose. For example, an existing service may have a set * ' 
sample size of 1,000 chosen to represent a state with an adult population of 
10 million — not an unlikely sample size if carefully chosen, and may be 
adequate to reflect the entire state within a sampling error of less than 
three percent. But if an early question in the survey divides the sample 
into two or more response streams (sets of questions for subsamples of the 
respondents), the number of respondents in each stream may be too small to 
provide meaningful results that can be generalized to the whole population. 

Finally a fourth type of field study is a mix of activities, including 
canvassing representatives of groups of potential learners, or preparing a 
study design which involves unobtrusive measures — that is, a methodology 
that collects data in indirect ways. One of the criticisms often heard of 
typical surveys that are designed to predict behavior of respondents is that 
the level of actual response is rarely as high as the data would have one 
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expect. It appears that in some inst.iuces respondents are willing to agree 
with sentiments and opinions they perceive that researchers want to hear, and 

choose responses -accordingly . — 

If a research question is fairly simple and direct, one can sometimes 
measure opinion in creative ways that do not involve asking the respondent 
any questions, but simply observing the behavior of a sample of appropriate 
people in a given situation, people "voting with their feet," for example. 
The rate of viewer response to an educational offering over broadcast television 
may be one indicator of the likely response to an educational program 
utilizing broadcast instruction. The "trial balloon" and the "stalking horse" 
are both variations on this methodology, often used in the political arena. 

Educational researchers with limited means but a creative cast of mind may 

I 

find a number of instances were unobtrusive measures may be a useful and < 

9 

appropriate survey technique. 

The other approach — direct contact with representatives of groups of 
potential new learners — is one that is commonly used by adventurous community 
college personnel trying to be of greater service to their communities. It 
involves contacting employers, unions^ retired person's associations, and other 
agenci.3S which purport to represent an identifiable group. On occasion these 
contacts pay off, but according to the Director of the National Center for 
Educational Brokering, community learning needs are usually not tied in any 
direct way to organizations whose main purpose is riOt learning. 

The disadvantages of these methods are apparent. Usually, research needs 
are more complex and extensive than can be easily answered in a single 
test situation or a scattering of visits to community organizations. But in 
the fluid world of policy research, such irregular data, if convincingly 
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prepared, will often be sufficient to buttress the arguments for new programs 
almost as effectively as a more conventional survey strategy. 

Strengths and weaknesses 

These sketches of types of survey techniques are not intended to be 
cunn)rehensive, nor to carry with them all the considerations that must be made 
when undertaking a serious research design. For that one should contact 
either a professional polling firra, an experienced survey research specialist, 
or consult one or another of the excellent texts on sampling, surveying, and 
statistical analysis, But the several types of survey methods are suggestive 
of the techniques which have been used by various research analysts who have 
conducted studies helpful to planners and policy makers at the state and 
local levels. I 

The requirements of the study and its intended uses usually imply the 
appropriate type field survey to use. It is important to realize, however, 
that all choices in survey research involve a trade«off among considerations 
such as cost, speed, simplicity, scientific accuracy, nurcher and complexity of 
questions, sample size and composition, or level of research competence. A 
quick-and-dirty study may seem to be as effective as aa elegant and expensive 
design, but the loss in quality may have some unforeseen consequences: critics 
can more easily discount the data by devastatingly exposing its shortcomings 

Professional polling firms are more likely to. generate unimpeachable 
results (albait shrouding their methodology somewhat) but often fail to follow 
through with the needed help in planning how best to use the information. 
It is our opinion that in most cases institutions which can draw upon their own 
staffs or knowledgeable educational consultants for studies of the kind 
presented her^i should do so^ perhaps contracting for an interview or telephone 
survey with a polling firm, but maintaining control of the project "in house." 
In most cases this proves to be the most satisfactory, 
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The strength, then, of survey research Is to enable planners and policy 
makers to answer basic questions about the need for, probable impact of, and 
opinions about new educational resources. With care in- design, some very 
specific questions can be answered, such as: "How ziny adults in our city 
are likely to sign up in the first year of an individualized, part-time, 
home study program leading to the Bachelor of Liberal Studies degree; how 
much will they spend; how many hours a week can they study; and what supporting 
services will they need in order to successfully enter and remain in the 
program?" This sort of question is not untypical of those in other studies. 

There are, however, weaknesses in relying exclusively on field polls 
to derive planning data. As we well know from political polls, opinions 

within the population change, often quite rapidly. Moreover, polls are usually 

I 

of one-^shot design not intended to give an idea of trends. Research on adult ' 
learners shows several typical characteristics which need to be taken into 
account in any planning situation: adults are generally interested in learning 
which has immediate utility (not for some future reward) and they will have 
shifting needs and demands year by year • The same person may need vocational 
skills one year, child development information the next, an introduction to 
the humanities and a third, and community affairs the fourth. Along with each 
of these felt needs the learner will want information and nondirective . 
guidance, and perhaps financial aid. In all cases he or she will demand to 
be treated as an adult who has active managment over his educational experiences. 
Thus planners will need to arrange some systematic means of continually 
monitoring the size of the demand for various learning opportunities. 

Field surveys also ignore the needs of manpower planning for industries, 
businesses, and trade unions. The need for skills and abilities for the economic 
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well-being of a city may not be perceived by the population in general, and may 
not show up in an inventory of learning needs. Moreover, surveys are an inconstant 
tool with some of the most disadVanti^^^^ of ' die TopulaTion who are " " 

not accustomed to polling and may not wish to express true feelings to an 
interviewer. Research studies show without a doubt that the generally well 
off, more highly educated, middle-class citizen is the one interested in more 
education; the poor, the under educated, and the underprivileged either do net 
show up in the data or- characteristically express a negative reaction to questions 
about further education. 

Third, field surveys sometimes ignore the strength of resources an 
Institution may possess in certain fields of study or at certain levels of 
education. If you are in a liberal arts college your faculty may vlp with 
alarm the poll results that show a great need for training in industrial trades'. 

Finally, survey research is a tricky business. Questions which seem 
perfectly clear to the researcher sometimes prove to be disastrously vague 
and ambiguous to many respondents. Questionnaire design is an art and a science' 
in itself, and when used in a carefully developed strategy can yield useful, 
reliable results. Sloppy work in developing either one will lead only to 
embarrassment: garbage in— garbage out. Without either selecting proven 
questions borrowed from previously successful instruments, or carefully pretesting 
new questions on a sample of respondents, the researcher runs the considerable 

> 

danger of confusing, or worse, offending his subjects. These respondents may 
either reject the whole effort or begin deliberately falsifying replies to 
get back at the offense. In either case, the study may be invalidated. 

The business of choosing an approach methodology, selecting a sample, 
and determining what constitutes an appropriate sample size to yield what 
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level of accuracy is needed for the study are all important questions that 
must be correctly answered in order for the study to result in maximum pay-off. . 
If the choice rests on igoihg with a locally developed "strategy which 
there is some question, or bringing in a survey research specialist to put 
it right, don't hesitate to bring in the specialist. The added cost will 
likely be well worth it. 

Prospects for improved studies 

It is clear that there are benefits to be derived from determining educational 
needs through survey research, but there are also drawbacks. Other efforts 
will also have to be employed if planners and policy boards are to. have 
sufficient information at their command to plan for the future. The evidence 
is that sophisticated use of field surveys is increasing as educatiorial plannerfi 
become more accustomed to this research tool. Of the 47 studies referred to 
earlier, the last 20 are, in general, much better designed and executed than 
Che first 20. We hope and expect that this trend will continue as we struggle 
to improve the methods of our profession. 
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